What happens when a logistics system has to handle very different data loads at the same time?
In this project, data from different sources was processed synchronously, creating uneven loads on the database. During peak workloads, this could slow down or even block the database and disrupt systems that depended on it.
We introduced an intermediate Python service and Kafka. NiFi sends messages to the appropriate Kafka topic, while our service retrieves them in batches, applies business rules, and loads the results into the database. This made the workload more predictable and the architecture easier to scale.
The same data bus was later reused for two more tasks: integrating a new B2B customer through data mapping and building a public shipment tracking service with its own database.
The interesting part wasn't just adding Kafka. It was finding a way to stabilize the existing data flow and then reuse that infrastructure for new integration scenarios.
Full technical case:
https://evrone.com/cases/edi-for-logistics
In this project, data from different sources was processed synchronously, creating uneven loads on the database. During peak workloads, this could slow down or even block the database and disrupt systems that depended on it.
We introduced an intermediate Python service and Kafka. NiFi sends messages to the appropriate Kafka topic, while our service retrieves them in batches, applies business rules, and loads the results into the database. This made the workload more predictable and the architecture easier to scale.
The same data bus was later reused for two more tasks: integrating a new B2B customer through data mapping and building a public shipment tracking service with its own database.
The interesting part wasn't just adding Kafka. It was finding a way to stabilize the existing data flow and then reuse that infrastructure for new integration scenarios.
Full technical case:
https://evrone.com/cases/edi-for-logistics