Migration to a Cloud-Based Data Fabric - Sparebank 1

USAGE OF A MEDALLION ARCHITECTURE WITHIN A DATA FABRIC

A Data Fabric is an architectural approach that utilizes metadata-driven automation and intelligent capabilities to create a unified and consistent data management layer across diverse and distributed data environments, enabling seamless data access and delivery.

The Medallion Architecture organizes the data in a data platform into three layers: Bronze, Silver, and Gold, each representing increasing levels of data refinement and quality. This approach allows for incremental improvements in data quality as it flows through the layers. The Bronze layer represents the raw, unfiltered data ingested from various sources. The Silver layer contains cleansed and transformed data from the Bronze layer. The Gold layer represents the final, curated data ready for business analysis and reporting.

Almost everything within the Data Fabric’s Silver layer got modeled with the Data Vault 2 modeling approach. Yes, just “almost”. Not every Team went through the core Data Vault, but their solutions are still part of the whole Data Fabric using its governed standards. Speaking in the Medallion Architecture: Only the silver layer was different in a few cases. Those teams used the same Bronze layer as a foundation and the same Gold layer definition as the presentation layer by following the same governed and automated processes as any other team.

The main reason why some teams didn’t use the Data Vault modeling approach was that the data objects they handled were either not directly related to what was defined as the “core”, or Data Vault knowledge was not enough available in each team. This decision allowed these teams to move forward, producing data products without becoming a bottleneck, but still following the overall data governance rules.

SUCCESS STORY: MIGRATION TO A CLOUD-BASED DATA FABRIC - PEOPLE, TECHNOLOGY & METHODOLOGY

5/12

Powered by