Streamlining data engineering operations for an international sports apparel manufacturer.
Service
-
Engineering
Our role
We did it. Building a platform for the technical development team to deliver a smooth package of tools to for the rest of the organization that will benefit from clear data. This project turned out to be one our most technical ones thus far, and reminded us why high scalable data infrastructure is crucial for future growth.
The challenge
Our challenge was to funnel multiple business processes worldwide with insights and dashboards. In order to leverage as much as possible on the created dashboards, the development team wanted to standardize all the data engineering related efforts, so all data engineers could create insights as fast as possible.
The solution
We helped the customer with the realization of an internal data platform that allowed all data engineers in the company to create data pipelines with just simple configuration files. The platform was built with open-source tooling only that allowed them to create a pipeline in just a few minutes. In the background it would create a Spark cluster, and Airflow scheduler so data was processed at a massive scale, without needing to understand all the details.
Blenddata supported the realization of new features, but also created a testing framework, which increased the deployment velocity.
The outcome
The biggest win of this project was clear: The time to create a new pipeline in production was significantly decreased to days instead of weeks. Where teams were inventing the wheel over and over again, they can now tap into the 100+ best practices of the entire company.