Here you can find out about …
Intelligent data management
- Data warehouse structure – timeless, sustainable, highly adaptable
- Data governance – very simple, very logical
- DSGVO / GDPR secure implementation in the data warehouse
also:
- How do we easily find the correct and complete data?
- How can we become faster with our data?
- Why is everyone talking about “metadata”?
A good data warehouse structure is timeless, sustainable and highly adaptable
Technology changes, systems change, business models change. A good data warehouse adapts flexibly to these changes and continues to fulfill two important characteristics:
- Historization – within the changing environment, the DWH ensures that the historicized deliveries fit together seamlessly
- Integration – the company is (hopefully) not structured according to operational systems and data sources but is based on a business model with business objects, transactions, hierarchies and relationships that are brought together from the various sources.
Intelligent data management fulfills these characteristics regardless of the source systems and data platforms used.
Note: Data Vault 2.0 as a methodology for intelligent data management provides concepts that can be used in virtualized or materialized form, which are open for structured (RDBMS tables), semi-structured (XML, JSON, Avro, …) or unstructured (texts, photos, videos, …) data.
A common misconception is that data needs to be converted and stored multiple times. The Data Vault 2.0 methodology is open in this respect, which makes it timeless.
Problem child or “challenge” of data governance – quite simply, quite logically
In the past, data governance was often a problem child because governance structures (responsibility for a data object such as “customer”, “product”, “order”, “production”) differed greatly from data structures in the source system and in the data warehouse. Data Vault 2.0 organizes the data in the data warehouse in exactly the same way as data governance (tools). Data Vault 2.0 is easy to automate and there are now many good tools for this. For every budget, level of automation and depth of automation. Automation implicitly automates the lineage from the source to the report, the connection from the data vault to the (conceptual) governance model is almost 1:1. If you focus on these principles, efficient governance is easily possible.
As few brakes as possible: DSGVO / GDPR secure implementation in the data warehouse
DSGVO / GDPR is a topic that is still weighing heavily on the minds of many companies. If it seems to have been tackled and dealt with well on the operational side, there are completely new challenges in the data warehouse. The long-running historization, combined with schema evolution and perhaps a lot of data that does not come from operational source systems, all want and need to be taken into account. This applies to the data warehouse as well as the lake or the PSA (Persistent Staging Area).
How do we easily find the correct and complete data?
You have many sources of data in your company. Nowadays, you also have a lot of structured, digital data. Your applications are often based on databases, whether in your data center (“on-prem”) or in the cloud.
These applications help you in your day-to-day business and support your processes, such as finance, ordering and order processing, time recording, document management and much more. The disadvantage of these applications: They deal with the here and now, historical changes to data are typically not saved. Only with this “historization” can you answer some important questions, such as
- What was the customer’s previous address?
- How many and which changes are first recorded in quotations before an order is placed?
- etc.
For this type of historization, the data is extracted from the application databases and placed in a data pool. Until a few years ago, this was the data warehouse, then the data lake was added.