Data vault modeling

Summary

Data vault modeling is a database modeling method that is designed to provide long-term historical storage of data coming in from multiple operational systems. It is also a method of looking at historical data that deals with issues such as auditing, tracing of data, loading speed and resilience to change as well as emphasizing the need to trace where all the data in the database came from. This means that every row in a data vault must be accompanied by record source and load date attributes, enabling an auditor to trace values back to the source. The concept was published in 2000 by Dan Linstedt. Data vault modeling makes no distinction between good and bad data ("bad" meaning not conforming to business rules). This is summarized in the statement that a data vault stores "a single version of the facts" (also expressed by Dan Linstedt as "all the data, all of the time") as opposed to the practice in other data warehouse methods of storing "a single version of the truth" where data

Official source

https://en.wikipedia.org/wiki/Data_vault_modeling

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (16)

Related publications

Related people (4)

Related people

Related units (2)

Related units

Related concepts

Related courses (5)

This course is intended for students who want to understand modern large-scale data analysis systems and database systems. It covers a wide range of topics and technologies, and will prepare students to be able to build such systems as well as read and understand recent research publications.

This course gives an introduction to the fundamental concepts and methods of the Digital Humanities, both from a theoretical and applied point of view. The course introduces the Digital Humanities circle of processing and interpretation, from data acquisition to new understandings.

This course provides a deep understanding of the concepts behind data management systems. It covers fundamental data management topics such as system architecture, data models, query processing and optimization, database design, storage organization, and transaction management.

Related courses

Related lectures (4)

Related lectures

TELEIOS: A Database-Powered Virtual Earth Observatory

Manolis Karpathiotakis, Ying Zhang

TELEIOS is a recent European project that addresses the need for scalable access to petabytes of Earth Observation data and the discovery and exploitation of knowledge that is hidden in them. TELEIOS builds on scientific database technologies (array databases, SciQL, data vaults) and Semantic Web technologies (stRDF and stSPARQL) implemented on top of a state of the art column store database system (MonetDB). We demonstrate a first prototype of the TELEIOS Virtual Earth Observatory (VEO) architecture, using a forest fire monitoring application as example.

2012

NoDB: Efficient Query Execution on Raw Data Files

Anastasia Ailamaki, Ioannis Alagiannis, Miguel Sérgio De Oliveira Branco

As data collections become larger and larger, data loading evolves to a major bottleneck. Many applications already avoid using database systems, e.g., scientific data analysis and social networks, due to the complexity and the increased data-to-query time. For such applications data collections keep growing fast, even on a daily basis, and we are already in the era of data deluge where we have much more data than what we can move, store, let alone analyze. Our contribution in this paper is the design and roadmap of a new paradigm in database systems, called NoDB, which do not require data loading while still maintaining the whole feature set of a modern database system. In particular, we show how to make raw data files a first-class citizen, fully integrated with the query engine. Through our design and lessons learned by implementing the NoDB philosophy over a modern DBMS, we discuss the fundamental limitations as well as the strong opportunities that such a research path brings. We identify performance bottlenecks specific for in situ processing, namely the repeated parsing and tokenizing overhead and the expensive data type conversion costs. To address these problems, we introduce an adaptive indexing mechanism that maintains positional information to provide efficient access to raw data files, together with a flexible caching structure. Our implementation over PostgreSQL, called PostgresRaw, is able to avoid the loading cost completely, while matching the query performance of plain PostgreSQL and even outperforming it in many cases. We conclude that NoDB systems are feasible to design and implement over modern database architectures, bringing an unprecedented positive effect in usability and performance.

2012

, ,

The ability of enterprise managers to come up with the decisions leading to the best outcome for the enterprise is hampered by their cognitive limits in understanding and addressing the dynamic and structural complexities residing in choice situations. Dynamic complexities deal with the behavior of the enterprise and its environment over time. Structural complexities, on the other hand, arise from the number of the departments within the enterprise, their interactions and the interactions between the enterprise and the entities across its boundary. The policy aiding methods developed to assist managers in the analysis of choice scenarios address these two types of complexities in separation. In this paper adopting a holistic approach, we integrate System Dynamics (SD) - a method for understanding the behavior of systems over time - and Systemic Enterprise Architecture Methodology (SEAM) - a modeling method that provides insights into how an enterprise and its interactions with other entities are structured. Integrating SD and SEAM, we present an approach to modeling, analysis and simulation of choice scenarios aiming at reducing the dynamic and structural complexities involved in the decision making process. We illustrate the applicability of our approach by applying it to an example of a choice situation in a manufacturing company.

2010