Two storage architectures, one catalog
The company publishes its product line to customers on a public portal and exchanges electronic documents with partners — both driven by a product catalog held in a MongoDB document database. The products themselves are managed in an ERP system on Microsoft SQL Server, where each product’s description, attributes, and component list are spread across many normalized relational tables.
Keeping the catalog current meant bridging two storage architectures, not just two systems. A single catalog document aggregates rows from many ERP tables, so the obvious approach — regenerate the whole document whenever anything upstream changes — is expensive, floods the portal and partner feeds with unchanged data, and loses track of what actually changed.
The company needed catalog documents that follow ERP changes continuously, at the granularity of the change itself, without batch regeneration jobs or extra load on the ERP database.
Per-table events, field-level updates
A2 built an event-driven replication pipeline on the ERP’s native change capture. Changes to the relational tables are tracked with Microsoft SQL Server CDC and Change Tracking; for each changed table the pipeline generates a message describing exactly what changed and publishes it to Apache Kafka.
On the consuming side, every message is mapped onto the part of the MongoDB document that derives from the changed table — and only that part. An update to the table holding product descriptions changes only the description field of the product document; a change to one component of a product updates only that component’s entry inside the document. Documents are never rebuilt wholesale, so the portal and partner exchange receive precise, incremental updates.
The result is replication not just between heterogeneous systems but between different storage architectures — relational and document — with the granularity of each change preserved end to end. The same pattern is now productized: SQL Server sources and incremental MongoDB document targets are available in Gearacles, the commercial edition of oracdc.
Client identity is withheld under a non-disclosure agreement. Engagement details can be discussed under NDA where appropriate.
Facing a similar challenge?
Talk to the team that built this; we'll discuss your environment, not a generic slide deck.