CHALLENGES
One issue is that there are cases in which the reported information is inconsistent across the frontends: while each system by itself provides correct information, the measurements are not comparable, for example because they have been collected at different times or under different circumstances. On the one hand, none of the warehouse developers were trained programmers nor was there an underlying scheme for the warehouse. On the other hand, what was lacking in programming skills was made up for in technical knowledge of steel production. This concept worked utilizing an average of two people for quite some time until the individuals retired. The next generation of warehouse developers faced largely undocumented code, some of which was difficult to read. In some places, it was unclear which version of the code was productive, making debugging, maintenance and enhancement very difficult. In addition, technical knowledge about the production process that people had previously acquired
over years was missing. Adjustments to the ETL applications were like playing Mikado, if you made changes in the wrong place it could break other ETL routes as well. In addition to the aforementioned challenges with the existing solution, it is foreseeable that it is not possible to implement new functionality that may be required in the future. For example, Dillinger is currently in the process of establishing data quality and data governance initiatives. Also, it should be possible to centrally define and manage key performance indicators (KPIs) and business logic. In some cases, there is also no meta-data available for the source datasets and the processing of business logic in the data warehouse. In short, the current data warehouse solution is unable to meet the requirements from these initiatives. Also, advanced topics like Data Science Support or Predictive Maintenance, are not well supported. It was time to rethink the old concepts and build a modern data warehouse with a promising future.
SOLUTION APPROACHES
For Dillinger, it was important that any new system should also be maintainable utilizing only a few people. Dillinger’s main business is steel production and their IT department is limited in size. Another requirement was that the new solution should be built in an incremental fashion. As the old warehouse has over thirty years of development, a big bang solution would be very risky and require a lot of manpower. They also wanted to have a scalable data warehouse as it should be possible to start with a small setup in terms of hardware and number of source systems and increase as needed. This should also include the selection of database technologies: ● They wanted to be able to integrate different technologies and the new solution should be flexible enough to allow it. ● In order to have a future-proof concept in place, Big Data technologies should also be supported since these will be required at some point. ● Additionally, Self-Serviced BI was a further requirement. ● Departments should be able to quickly create their own evaluations based on warehouse data.
Made of Steel - Data Vault 2.0 at Dillinger | Scalefree International GmbH
4/13
Powered by FlippingBook