JIT works: decide when all data is known (VLDB Women in Database Research Award talk)

Anastasia Ailamaki
ASSOC COMPUTING MACHINERY, 2020
Article

Résumé

When the time comes to make a critical decision, it is of paramount importance to prepare enough so that all the information necessary is available at decision time. Under-preparation leads to uninformed decisions; over-preparation, however, may lead to conservative decisions. Often a decision seems suboptimal after its consequences are known; regret is a typical reaction in such a situation. In life and in my research I have often followed the wrong path. More accurately, I have often followed a path that led to suboptimal results or unwanted consequences. The difficulty is to fight the urge to regret and rather learn from the process; divide the input that led to the wrong path into the part that could have been made better (and is the reason for the suboptimal outcome) and the part that was already as good as it gets. Next time, prepare less for the former while keeping the latter an invariant. We are often faced with daunting decisions when designing database systems on modern heterogeneous, ever-faster hardware platforms. After having built hierarchical, object-oriented, relational, and just-in-time (JIT) systems by myself or with a team, I realize that a great deal of the performance is determined less from determining what data model to use or what index to build ahead of time and more from making decisions at the right time. Educated JIT works using mathematics and dynamic programming, and it helps prepare for invariants. We make decisions only when all the information is at hand for the more flexible (and more decisive) part of the input. Even if the response time of the first few requests is high, these serve as "teaching" exercises for the subsequent ones. Tools like code generation and data virtualization help make JIT useful in data management. I hope that our community will build many JIT systems and benefit from the dynamic nature of this philosophy. As many computer scientists do, I routinely use what I learn in my research to optimize performance in a multi-dimensionally unpredictable everyday life, and vice versa. I've built systems by mapping design choices to life challenges, and have tried to relate to my students' personalities and help them overcome hurdles while filtering through my (often boring, but hopefully useful) personal experiences. In my talk I will share some of these experiences, intertwined with exciting research directions. I am humbled and grateful to receive the VLDB Women in Database Research award, and I thank the community for this distinction.

Official source

https://infoscience.epfl.ch/record/282808?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Anastasia Ailamaki
ASSOC COMPUTING MACHINERY, 2020
Article

Résumé

Official source

https://infoscience.epfl.ch/record/282808?ln=fr

À propos de ce résultat

Concepts associés (24)

Concepts associés

Chargement

Publications associées (20)

Publications associées

Chargement

Adaptive Query Processing on RAW Data

Anastasia Ailamaki, Ioannis Alagiannis, Miguel Sérgio De Oliveira Branco, Manolis Karpathiotakis

Database systems deliver impressive performance for large classes of workloads as the result of decades of research into optimizing database engines. High performance, however, is achieved at the cost of versatility. In particular, database systems only operate efficiently over loaded data, i.e., data converted from its original raw format into the system’s internal data format. At the same time, data volume continues to increase exponentially and data varies increasingly, with an escalating number of new formats. The consequence is a growing impedance mismatch between the original structures holding the data in the raw files and the structures used by query engines for efficient processing. In an ideal scenario, the query engine would seamlessly adapt itself to the data and ensure efficient query processing regardless of the input data formats, optimizing itself to each instance of a file and of a query by leveraging information available at query time. Today’s systems, however, force data to adapt to the query engine during data loading. This paper proposes adapting the query engine to the formats of raw data. It presents RAW, a prototype query engine which enables querying heterogeneous data sources transparently. RAW employs Just-In-Time access paths, which efficiently couple heterogeneous raw files to the query engine and reduce the overhead of traditional general-purpose scan operators. There are, however, inherent overheads with accessing raw data directly that cannot be eliminated, such as converting the raw values. Therefore, RAW also uses column shreds, ensuring that we pay these costs only for the subsets of raw data strictly needed by a query. We use RAW in a real-world scenario and achieve a two-order of magnitude speedup against the existing hand-written solution.

2014

Chargement

Here are my Data Files. Here are my Queries. Where are my Results?

Anastasia Ailamaki, Ioannis Alagiannis, Frederick Ryan Johnson

Database management systems (DBMS) provide incredible flexibility and performance when it comes to query processing, scalability and accuracy. To fully exploit DBMS features, however, the user must define a schema, load the data, tune the system for the expected workload, and answer several questions. Should the database use a column-store, a row-store or some hybrid format? What indices should be created? All these questions make for a formidable and time-consuming hurdle, often deterring new applications or imposing high cost to existing ones. A characteristic example is that of scientific databases with huge data sets. The prohibitive initialization cost and complexity still forces scientists to rely on "ancient" tools for their data management tasks, delaying scientific understanding and progress. Users and applications collect their data in flat files, which have traditionally been considered to be "outside" a DBMS. A DBMS wants control: always bring all data "inside", replicate it and format it in its own "secret" way. The problem has been recognized and current efforts extend existing systems with abilities such as reading information from flat files and gracefully incorporating it into the processing engine. This paper proposes a new generation of systems where the only requirement from the user is a link to the raw data files. Queries can then immediately be fired without preparation steps in between. Internally and in an abstract way, the system takes care of selectively, adaptively and incrementally providing the proper environment given the queries at hand. Only part of the data is loaded at any given time and it is being stored and accessed in the format suitable for the current workload.

2011

Chargement

In recent years, funding agencies, governments, and international organization have issued a series of research data management and sharing policies, like the H2020 [1]. Researchers are encouraged or required to share their production in an open way, including attached publications and data [2]. One consequence of those policies for researchers is the necessity of providing a DMP (Data Management Plan) when applying for funding. This DMP describes the life-cycle of the data during the funded project. In particular, researchers have to specify a description of the generated data, along with their storage, protection, publication and archive strategies. In order to assist researchers with those new requirements and provide them with good practices, many university libraries have developed services around the management of their data [3a][3b]. Among the many information required in a DMP, some are dedicated to the costs the researchers will face for their data management, storage or curation. Those questions are usually painful, as the applicant has to have full knowledge of existing providers for storage, archive or publication services. These providers can be part of their institution or exist outside. Few helpful information exists. One can find costing tools in form of a spreadsheet or a static document in which the researcher is lost or will spend a lot of time to find the needed information. Most of these tools do neither give any providers’ nor real cost information. It's in this context that the Research Data Management Team of EPFL Library has developed an online tool, creating a central place for researchers to find and calculate their costs related to the data life-cycle [4]. The tool contains a cost database that includes most of our institution services and some external providers. Furthermore, this tool provides good practice for the researchers around calculating costs. It is designed entirely for self-use and doesn't need any training. By using a modern code repository and open source libraries, the cost database can be updated and changed easily without specific technical knowledge, which makes deployment in other institutions handy. The cost database can be implemented with any kind of providers, such as institutional or private ones and adapted for any kind of pricing model. During the presentation, we will first introduce the tool and its functionalities, discuss the advantages for the researchers, and end with how the tool can be used by other interested partners. Bibliography [1] : Guedj, D., Ramjoué, C.: European Commission Policy on Open-Access to Scientific Publications and Research Data in Horizon 2020. Biomed Data J. 2015;1(1):11-14. DOI: http://dx.doi.org/10.11610/bmdj.01102 [2] : Whyte, A., Pryor, G.: Open Science in Practice: Researcher Perspectives and Participation. Int. J. Digit. Curation. 6, 199–213 (2011). DOI : https://doi.org/10.2218/ijdc.v6i1.182 [3a] : Peters, C., Dryden, A.R.: Assessing the Academic Library’s Role in Campus-Wide Research Data Management: A First Step at the University of Houston. Sci. Technol. Libr. 30, 387–403 (2011). DOI : https://doi.org/10.1080/0194262X.2011.626340 [3b] : Tenopir, C. et al.: Research Data Services in European Academic Research Libraries, LIBER Quarterly, vol. 27, n° 1, 23‑44 (2017). DOI : http://doi.org/10.18352/lq.10180 [4] : Cost Calculator, A. Masson, EPFL Library DOI: http://dx.doi.org/10.5281/zenodo.1469035 https://rdmepfl.github.io/costcalc/

2019

Chargement

2019