AiiDA: automated interactive infrastructure and database for computational science

Computational science has seen in the last decades a spectacular rise in the scope, breadth, and depth of its efforts. Notwithstanding this prevalence and impact, it is often still performed using the renaissance model of individual artisans gathered in a workshop, under the guidance of an established practitioner. Great benefits could follow instead from adopting concepts and tools coming from computer science to manage, preserve, and share these computational efforts. We illustrate here our paradigm sustaining such vision, based around the four pillars of Automation, Data, Environment, and Sharing. We then discuss its implementation in the open-source AiiDA platform (http://www.aiida.net), that has been tuned first to the demands of computational materials science. AiiDA's design is based on directed acyclic graphs to track the provenance of data and calculations, and ensure preservation and searchability. Remote computational resources are managed transparently, and automation is coupled with data storage to ensure reproducibility. Last, complex sequences of calculations can be encoded into scientific workflows. We believe that AiiDA's design and its sharing capabilities will encourage the creation of social ecosystems to disseminate codes, data, and scientific workflows. (C) 2015 Elsevier B.V. All rights reserved.

Thermal transport in low dimensions

Andrea Cepellotti

Lattice vibrations are the microscopic mechanism responsible for a large, if not dominant, contribution to heat transport in crystalline insulators. These vibrations are described in terms of phonons, collective excitations (or quasiparticles) in the form of waves of atomic displacements inside a crystal. Phonons are traditionally considered to be the quasiparticles responsible for carrying heat through the material. Heat transport is considered as a flux of a phonon gas, diffusing from hot areas (high phonon densities) to cold areas (low phonon densities) in an attempt to reestablish equilibrium, with phonon collisions being the source of heat flux dissipation. However, as dimensionality is reduced, the motion of phonons stimulated by temperature perturbations becomes correlated and this gas-like picture of thermal transport in terms of phonons becomes invalid. In this Thesis, we lay out an interpretation of thermal transport in 2D materials based on the Boltzmann transport equation in the form of collective excitations of phonons. These collective phonon excitations give raise to complex phenomena, such as high thermal conductivities, that are otherwise unexplained. As another example, collective excitations, at variance with conventional diffusive transport, can induce wave-like heat propagation, or second sound. This had been found only in a few exotic materials at cryogenic temperatures, but is present instead routinely in 2D materials at room temperature. The correlated-phonon description of heat transfer can be rationalised by introducing a new collective excitation, called 'relaxon', which is defined as the eigenstate of the collision operator. Whereas only oversimplifying assumptions endow phonons with well defined relaxation times (the average interval of time between collisions), relaxons have always well defined relaxation times and permit an exact description of thermal transport. The complex dynamic of heat transport in 2D is thus greatly simplified and a kinetic gas theory of thermal transport still applies, provided that the gas is not constituted by phonons, but by relaxons. Our work on thermal transport is part of a larger effort, aiming at the creation of a database of numerically computed properties of materials. The high-throughput production of simulated properties is a challenging task, since it necessitates the understanding of a physical model, but it also needs to face a myriad of technicalities and problems that hinder the execution of a large number of calculations. In order to allow the creation of computational materials databases, we developed AiiDA, an open-source automated interactive infrastructure and database for computational science. This platform tackles the problems of creation, management, analysis and sharing of data and simulations, summarized in the pillars of Automation, Data, Environment and Sharing. Automation is achieved by management of remote computational resources and the encoding of workflows, that allows the execution of complex sequences of calculations. The tight coupling between automation and data storage, handled by the platform, enables full reproducibility of the results and a suitable database design allows for efficient data analysis tools. Sharing of scientific knowledge is addressed by providing tools for distribution of data and of the underlying workflows that generated them, creating an ecosystem for computational materials science.

EPFL2016

Toward timely, predictable and cost-effective data analytics

Renata Borovica-Gajic

Modern industrial, government, and academic organizations are collecting massive amounts of data at an unprecedented scale and pace. The ability to perform timely, predictable and cost-effective analytical processing of such large data sets in order to extract deep insights is now a key ingredient for success. Traditional database systems (DBMS) are, however, not the first choice for servicing these modern applications, despite 40 years of database research. This is due to the fact that modern applications exhibit different behavior from the one assumed by DBMS: a) timely data exploration as a new trend is characterized by ad-hoc queries and a short user interaction period, leaving little time for DBMS to do good performance tuning, b) accurate statistics representing relevant summary information about distributions of ever increasing data are frequently missing, resulting in suboptimal plan decisions and consequently poor and unpredictable query execution performance, and c) cloud service providers - a major winner in the data analytics game due to the low cost of (shared) storage - have shifted the control over data storage from DBMS to the cloud providers, making it harder for DBMS to optimize data access. This thesis demonstrates that database systems can still provide timely, predictable and cost-effective analytical processing, if they use an agile and adaptive approach. In particular, DBMS need to adapt at three levels (to workload, data and hardware characteristics) in order to stabilize and optimize performance and cost when faced with requirements posed by modern data analytics applications. Workload-driven data ingestion is introduced with NoDB as a means to enable efficient data exploration and reduce the data-to-insight time (i.e., the time to load the data and tune the system) by doing these steps lazily and incrementally as a side-effect of posed queries as opposed to mandatory first steps. Data-driven runtime access path decision making introduced with Smooth Scan alleviates suboptimal query execution, postponing the decision on access paths from query optimization, where statistics are heavily exploited, to query execution, where the system can obtain more details about data distributions. Smooth Scan uses access path morphing from one physical alternative to another to fit the observed data distributions, which removes the need for a priori access path decisions and substantially improves the predictability of DBMS. Hardware-driven query execution introduced with Skipper enables the usage of cold storage devices (CSD) as a cost-effective solution for storing the ever increasing customer data. Skipper uses an out-of-order CSD-driven query execution model based on multi-way joins coupled with efficient cache and I/O scheduling policies to hide the non-uniform access latencies of CSD. This thesis advocates runtime adaptivity as a key to dealing with raising uncertainty about workload characteristics that modern data analytics applications exhibit. Overall, the techniques introduced in this thesis through the three levels of adaptivity (workload, data and hardware-driven adaptivity) increase the usability of database systems and the user satisfaction in the case of big data exploration, making low-cost data analytics reality.

EPFL2016

Toward timely, predictable and cost-effective data analytics

Renata Borovica-Gajic

EPFL2016

Thermal transport in low dimensions

Andrea Cepellotti

EPFL2016