Supporting the Migration of a Legacy Warehouse

INTRODUCTION A telecommunications company aims to improve their data integration and data product delivery via modernization and is looking to Scalefree for guidance on implementing a Data Vault 2.0 architecture. CASE DESCRIPTION The client is a subsidiary company specializing in information technology and telecommunications. They are operating within the transportation sector, providing crucial services for the technical infrastructure of its parent company and its various business units and are looking to enhance their data-driven decision making. They aim to reduce operational costs by reducing time and personnel expenses spent on implementing and maintaining the complex source systems feeding into their legacy warehouse as well as reduce turnaround time on GDPR compliance requests. It was thus decided to modernize and migrate their existing warehouse within a specific timeframe. Their data team, while consisting of technically proficient and experienced developers, was unfamiliar with designing and implementing a Data Vault 2.0 architecture. As these team members are also involved in maintaining their current warehouse, coupled with a sluggish decision making process, meeting the migration deadline posed an additional challenge to the client. After reviewing the case, our recommended course of action was as follows: we advised for an agile approach with short two-week sprints to manage the project and conducted workshops on the swift integration of multiple sources, using Meltano to load into a data lake on AWS S3 buckets. We guided the team through the process of modeling and implementing their assets in a Data Vault 2.0 fashion with a focus on standardized, efficient, scalable patterns to reduce friction within the team, speeding up delivery time on new data products.

We also offered recommendations on how to deal with GDPR compliance and how to leverage automation tools like Scalefree’s Turbovault and dbt with the datavault4dbt package to minimize the team’s time spent on maintaining and expanding loading patterns, instead allowing them to focus on delivering better data products in an efficient way. We focused our efforts on transferring knowledge and equipping the team with the right toolset to get working on their own as fast as possible, allowing them to utilize their wealth of experience and technological knowledge towards faster data integration and a more efficient, cost-saving data product delivery by kickstarting their implementation of a Data Vault 2.0 style warehouse.

2/3