Here are my Data Files. Here are my Queries. Where are my Results?

Database management systems (DBMS) provide incredible flexibility and performance when it comes to query processing, scalability and accuracy. To fully exploit DBMS features, however, the user must define a schema, load the data, tune the system for the expected workload, and answer several questions. Should the database use a column-store, a row-store or some hybrid format? What indices should be created? All these questions make for a formidable and time-consuming hurdle, often deterring new applications or imposing high cost to existing ones. A characteristic example is that of scientific databases with huge data sets. The prohibitive initialization cost and complexity still forces scientists to rely on "ancient" tools for their data management tasks, delaying scientific understanding and progress. Users and applications collect their data in flat files, which have traditionally been considered to be "outside" a DBMS. A DBMS wants control: always bring all data "inside", replicate it and format it in its own "secret" way. The problem has been recognized and current efforts extend existing systems with abilities such as reading information from flat files and gracefully incorporating it into the processing engine. This paper proposes a new generation of systems where the only requirement from the user is a link to the raw data files. Queries can then immediately be fired without preparation steps in between. Internally and in an abstract way, the system takes care of selectively, adaptively and incrementally providing the proper environment given the queries at hand. Only part of the data is loaded at any given time and it is being stored and accessed in the format suitable for the current workload.

JIT works: decide when all data is known (VLDB Women in Database Research Award talk)

Anastasia Ailamaki

When the time comes to make a critical decision, it is of paramount importance to prepare enough so that all the information necessary is available at decision time. Under-preparation leads to uninformed decisions; over-preparation, however, may lead to conservative decisions. Often a decision seems suboptimal after its consequences are known; regret is a typical reaction in such a situation. In life and in my research I have often followed the wrong path. More accurately, I have often followed a path that led to suboptimal results or unwanted consequences. The difficulty is to fight the urge to regret and rather learn from the process; divide the input that led to the wrong path into the part that could have been made better (and is the reason for the suboptimal outcome) and the part that was already as good as it gets. Next time, prepare less for the former while keeping the latter an invariant. We are often faced with daunting decisions when designing database systems on modern heterogeneous, ever-faster hardware platforms. After having built hierarchical, object-oriented, relational, and just-in-time (JIT) systems by myself or with a team, I realize that a great deal of the performance is determined less from determining what data model to use or what index to build ahead of time and more from making decisions at the right time. Educated JIT works using mathematics and dynamic programming, and it helps prepare for invariants. We make decisions only when all the information is at hand for the more flexible (and more decisive) part of the input. Even if the response time of the first few requests is high, these serve as "teaching" exercises for the subsequent ones. Tools like code generation and data virtualization help make JIT useful in data management. I hope that our community will build many JIT systems and benefit from the dynamic nature of this philosophy. As many computer scientists do, I routinely use what I learn in my research to optimize performance in a multi-dimensionally unpredictable everyday life, and vice versa. I've built systems by mapping design choices to life challenges, and have tried to relate to my students' personalities and help them overcome hurdles while filtering through my (often boring, but hopefully useful) personal experiences. In my talk I will share some of these experiences, intertwined with exciting research directions. I am humbled and grateful to receive the VLDB Women in Database Research award, and I thank the community for this distinction.

ASSOC COMPUTING MACHINERY2020

Adaptive Query Processing on RAW Data

Anastasia Ailamaki, Ioannis Alagiannis, Miguel Sérgio De Oliveira Branco, Manolis Karpathiotakis

Database systems deliver impressive performance for large classes of workloads as the result of decades of research into optimizing database engines. High performance, however, is achieved at the cost of versatility. In particular, database systems only operate efficiently over loaded data, i.e., data converted from its original raw format into the system’s internal data format. At the same time, data volume continues to increase exponentially and data varies increasingly, with an escalating number of new formats. The consequence is a growing impedance mismatch between the original structures holding the data in the raw files and the structures used by query engines for efficient processing. In an ideal scenario, the query engine would seamlessly adapt itself to the data and ensure efficient query processing regardless of the input data formats, optimizing itself to each instance of a file and of a query by leveraging information available at query time. Today’s systems, however, force data to adapt to the query engine during data loading. This paper proposes adapting the query engine to the formats of raw data. It presents RAW, a prototype query engine which enables querying heterogeneous data sources transparently. RAW employs Just-In-Time access paths, which efficiently couple heterogeneous raw files to the query engine and reduce the overhead of traditional general-purpose scan operators. There are, however, inherent overheads with accessing raw data directly that cannot be eliminated, such as converting the raw values. Therefore, RAW also uses column shreds, ensuring that we pay these costs only for the subsets of raw data strictly needed by a query. We use RAW in a real-world scenario and achieve a two-order of magnitude speedup against the existing hand-written solution.

2014

Efficient Protocols for Enforcing Causal Consistency in Geo-Replicated Key-Value Data Stores

Kristina Spirovska

Modern large-scale data platforms manage colossal amount of data, generated by the ever-increasing number of concurrent users. Geo-replicated and sharded key-value data stores play a central role when building such platforms. As the strongest consistency model proven not to compromise availability, causal consistency (CC) is perfectly positioned to address the needs of such large-scale systems, while preventing some ordering anomalies. Transactional Causal Consistency (TCC) augments CC by providing richer transactional semantics, simplifying the development of distributed applications. However, achieving CC/TCC in an efficient manner is very challenging. In this thesis we introduce several protocols and designs for high performance causally consistent geo-replicated key-value data stores in several different settings. First, we present a new approach to implementing CC in geo-replicated data stores, called Optimistic Causal Consistency (OCC). By introducing a technique that we call client-assisted lazy dependency resolution, OCC makes it possible for updates replicated to a remote data center to become visible immediately, without checking if their causal dependencies have been received. We further propose a recovery mechanism that allows an OCC system to fall back on a pessimistic protocol to continue operating during network partitions. We show that OCC improves data freshness, while offering performance that is comparable or better than its pessimistic counterpart. Next, we address the problem of providing low-latency TCC reads under full replication. We present Wren, the first TCC system that at the same time achieves low latency by implementing nonblocking read operations and efficiently scales out by sharding. Wren introduces new protocols for transaction execution, dependency tracking and stabilization. The transaction protocol supports nonblocking reads by providing a transaction snapshot as a union of a fresh causal snapshot installed by every partition in the local data center and a client-side cache for writes that are not yet included in the snapshot. The dependency tracking and stabilization protocols require only two scalar timestamps, resulting in efficient resource utilization and providing scalability in terms of shards and replication sites, under a full replication setting. In return for these benefits, Wren slightly increases the visibility latency of updates. Finally, we present PaRiS, the first TCC system that supports partial replication and provides low latency by implementing non-blocking parallel read operations. PaRiS relies on a novel protocol to track dependencies, called Universal Stable Time (UST). By means of a lightweight background gossip process, UST identifies a snapshot of the data that has been installed by every data center in the system. PaRiS equips clients with a private cache, in which they store their own updates that are not yet reflected in the snapshot. The combination of the UST-defined snapshot with client-side cache enables interactive transactions that can consistently read from any replication site without blocking. Moreover, PaRiS requires only one timestamp to track dependencies and define transactional snapshots, thereby achieving resource efficiency and scalability in terms of shards and replication sites, in a partial replication setting.