The promise of data warehouse automation was also encouraging: ETL jobs should be generated from metadata. The generated model, likewise generated from metadata, should allow historization of data to capture changes to productive data and allow traceability and full auditability. The new data warehouse should provide reliable and reusable KPI values that increase the consistency of the final reports. To achieve this, each metric should be calculated in this location and made available to reporting and end users.
SELECTION PROCESS
To evaluate their possibilities in terms of data warehouse technology and tooling, Dillinger started three proof-of-concepts together with Scalefree for the new TDW2.0 initiative in 2017. The goal of these PoCs was to evaluate two data warehouse automation tools and one manual implementation to compare the different approaches. Ultimately, Wherescape was selected. To prove the feasibility of DV2.0, Scalefree analyzed the main source system and focused on the main business objects: orders, positions, bars, slabs and sheets. For these, the first DV model was built within the scope of the PoC then in which a first data mart was loaded. Dillinger immediately understood the value of DV and chose it as their EDW Framework for the so-called technical data warehouse 2.0. The PoC loading was done with an open source ETL tool but once WhereScape was installed, Dillinger asked WhereScape’s on-site support team to accurately migrate the ETL process to WhereScape. This ensured that the system was properly configured for the first milestone. Once the feasibility of the Data Vault 2.0 approach was established, together with Wherescape ETL automation, the Dillinger warehouse team had to raise the project budget. It took a while to gather all necessary signatures and to release the budget though. However, Wherescape RED and 3D were licensed in the beginning of 2019 and the TDW2.0 project could start once cleared.
The idea was to lay the foundation for the warehouse in collaboration with Scalefree experts and train the internal team in parallel resulting in a continuing support and commitment from Scalefree.
Dillinger calculated internal and external costs in such a way that the project was able to run for six sprints together with Scalefree over the course of half a year.
PROJECT PROGRESS
The data warehouse development and implementation was steered by Scalefree and was planned to run with three internals and four externals on the long run. Six sprints, each of a one month duration, were planned. So, as indicated by the term “sprint”, the agile delivery, scrum-based approach was used for running the project. This is perfectly in line with the Data Vault 2.0 methodology and focuses on delivering a dashboard or at least part of a dashboard at the end of each sprint.
Made of Steel - Data Vault 2.0 at Dillinger | Scalefree International GmbH
5/13
Powered by FlippingBook