Migration to a Cloud-Based Data Fabric - Sparebank 1

A Testing environment will be started as soon as a new feature or fix is ready for deployment. Actually, the same procedure as in the development will happen here again, but in this case, completely automated and in a freshly created environment. This ensures that the changes will also work in the Production environment. As soon as deployed, the development and testing environment will be shut down again.

Adopting these agile development methods made it possible to create a collaborative and iterative environment for software development. This approach encourages continuous improvement through regular feedback, accelerates feature delivery via smaller tasks, enhances adaptability to evolving requirements, and boosts team productivity and morale through transparent, empowering processes. IMPROVEMENTS OVER TIME The starting point was a low-budget approach that mostly relied on open-source tools, self- written code, and custom processes. However, recognizing the limitations in scalability, a strategic transition to more robust, often paid, alternatives was made. This led to a migration from dbt Core to dbt Cloud, significantly easing the workload for business analysts and data engineers, replacing the self-developed DevOps pipelines, and leveraging numerous features that reduced development and deployment cycle times.

Furthermore, a shift occurred to dltHub for loading data into the Persistent Staging Area (PSA), incorporating Change Data Capture (CDC) mechanisms.

SUCCESS STORY: MIGRATION TO A CLOUD-BASED DATA FABRIC - PEOPLE, TECHNOLOGY & METHODOLOGY

8/12

Powered by