A flexible framework for efficient design of advanced 3D audio systems in multimedia applications

The recent advance of computer technology pushed the computing power of the modern computer systems to unprecedented levels; modern processors moved from the conventional, scalar type of architecture to more sophisticated, parallel ones, combining fast processing speed clock with multiple processing units. This jump of the computing power made possible the real-time implementations of many 3D audio algorithms that was hard to imagine 10-15 years before; on the other hand, the permanent increase of customer demand for real-time manipulation of the multimedia content leads to the development and implementation of more complex, and respectively more computationally demanding algorithms. The goal to deliver such sophisticated solutions to the market in a short time is already a difficult task; to achieve efficient solutions in such short terms, squeezing the maximum of the computing power of the currently available technologies, is even more challenging. In my thesis, I propose a solution to this problem by designing and implementing a framework that allows the developer to quickly design, implement and test custom 3D audio algorithms in an efficient way, with minimum efforts to port the already available code to another platform. The proposed approach started from a preliminary technology review and analysis in terms of development tools to find out and evaluate the state-of-the-art from 3D audio point of view. However, the performed analysis revealed some important drawbacks of these solutions in terms of fast reconfigurability for the implementation of custom 3D audio algorithms, and their capability for a precise synchronization and management of multiple stream media applications. First, it is in some cases necessary to develop completely new code to achieve the desired results, due to a lack of flexibility in the algorithm configurations or to unavailable functionality. Secondly, if the integration of different solutions and APIs, running at different hardware and software levels is pursued, this causes some problems with the synchronization between them, since high level APIs often introduce high latencies. The result is in some cases the system instability with consequent degraded quality, or even loss, of audio content for live and networked scenarios. In the end, it was noticed that if the APIs for 3D sound are higher level and developer-friendly they normally bring with them high latencies, sometimes-rigid 3D model solutions or high portability costs. If on the other hand they are lower-level solutions conceived for platform-independent low latency development and true real-time performance, the development of existing and particularly new 3D models may result long if good and robust source code is not available. To improve the situation, a new approach has been conceived to develop an independent middle-to-low level DSP library, which can provide a mid-layer development tool to easily design and configure new 3D algorithms. This simplifies at the same time the implementation task by providing the main building blocks in the field of signal processing and media streams, leaving to the developer enough flexibility and reconfigurability to implement the 3D audio models in a short time with the desired precision and quality. An object-oriented, two-level hierarchical structure has been adopted for the library. As a root of the hierarchy, some common properties for all primitives have been defined (e.g. number of input/output channels, single sample and block processing functions, etc.) and assigned to an object, common for all the DSP primitives (Parent Node). The first level (Atomic Level) contains the DSP primitives that cannot be built by the others: multiplier, adder, multiply-add, delay line, etc. Ideally, this layer corresponds to an instruction set and memory structures of a virtual machine designed for signal processing. The second level (Basic DSP Level) contains generic DSP primitives, e.g. low-pass filter, all-pass, nested all-pass, double nested all-pass, generic FIR and IIR filters, geometric and trigonometric operators (on vectors), FDN matrices, etc. This level is build by primitives, defined in the Atomic Level. Finally, as a test application, a 3D Audio layer has been implemented containing 3D algorithms, compatible first with the standardized VRML and MPEG-4 geometrical and perceptual room models, and in addition with several other algorithms like: Schroeder reverberator, Dattorro network reverberator, Gardner reverberator, etc. This allowed the successful integration of the proposed 3D audio library in a MPEG-4 compatible player, which has been implemented within the scope of the CARROUSO multimedia project. CARROUSO addressed the problem of how to transfer sound fields, generated at a certain real or virtual space, to another usually remote located space. The activities in CARROUSO did concentrate on the development and implementation of key components for recording, encoding, transmission and rendering, including hardware, software and algorithms. The transmission and coding was based on MPEG-4 and digital video broadcasting (DVB) technology, while sound recording and reproduction was based on wave field synthesis (WFS) technology. In this context, the MPEG-4 player is essential part of the chain, displaying the video content of the virtual scene, and providing the interface between the decoded sound content and the WFS rendering system. In contrast to the state-of-the art multi-channel sound systems, the new system has an enlarged listening area and is more flexible in respect to the number and the configuration of loudspeakers. The innovative work in CARROUSO found later its logical continuation in the MAP project where the developed 3D API has been exploited to integrate custom or standardized 3D audio models into a professional sound production toolbox, running on mixed general-purpose/FPGA/multimedia-DSP technology to obtain true real-time performance. In addition to the possibility to monitor locally multi-channel output, this tool provides at the same time the functionality to generate/encode object-oriented audio content, compatible to MPEG-4 BIFS format, which turns it into a sophisticated MPEG-4 authoring tool. As a conclusion, it could be noted, that the four years research and work during the CARROUSO and MAP projects, have proven in practice that the proposed 3D audio development framework is not only flexible and easily configurable, but it can be also very efficient, and ported to different platforms without putting significant effort in this task.

Easy and Accurate Hardware-based Program Performance Monitoring

Andrzej Pawel Nowak

The performance monitoring of computer systems is a complex affair, made even more challenging by the increasing gap between hardware and software. Methods that collect and feed data to performance analysis can usually be classified into one of two groups. Methods in the first group (e.g., software instrumentation), can be accurate but have unacceptable overheads and offer no insight into the interaction of the software and the hardware. Methods in the second group (e.g., hardware supported) run in real time and connect code to the response in the architecture, at a considerable decrease in accuracy, with a high degree of specificity to the hardware and therefore with limited actionable information made available. In this work, we focus on the latter group - the collection and analysis of raw performance data sourced from hardware Performance Monitoring Units, built into the majority of processors. The periodic collection of samples (Event Based Sampling), containing code locations, for instance, can be triggered on events such as retired instructions or cache misses. We quantify accuracy improvements to existing methods, propose a better performing method of performance data collection and a new architecture-agnostic method of performance data analysis. In our experiments, we analyze compiled, large-scale production workloads, industry standard benchmarks, and micro-benchmarks reproducing specific architectural behaviors. First, we focus on performance data collection methods and their accuracy, when establishing instruction retirement rates. These vary from least to most advanced and we examine the low level factors influencing accuracy. In particular, we employ a method based on Last Branch Records, a hardware facility present on some Intel architectures, which allows for the sampling of short records of last taken branches leading up to the sample. We compare the most commonly used method, its improved derivatives and the Last Branch Record based method. With respect to the first method, we observe accuracy improvements of up to 18x on synthetic benchmarks and 15x on real applications. Second, to further improve accuracy, we propose a new collection method named Hybrid Basic Block Profiling, fusing those based on Last Branch Records and the widely used Event Based Sampling. We apply simple machine learning techniques to determine the operating criteria. As a demonstration of capability, our method and a profiling tool are used to generate Instruction Mixes, which traditionally require good quality data at the most detailed level of granularity - basic blocks. Compared to software instrumentation, we observe an improvement in runtime of up to 76x, while keeping instruction attribution errors at 2.1% on average. In our third contribution we describe an architecture-agnostic performance analysis methodology called Hierarchical Cycle Accounting. The user is presented with a navigable hierarchy of microarchitectural issues, expressed in a metric that is universal and simple to compare - core cycles. We develop a reference implementation for the Intel Ivy Bridge microarchitecture and provide pointers for implementations on other processor architectures. Our approach and tool are successfully used by non-experts to improve and vectorize complex scientific code. Experts benefit as well - by being able to pinpoint software issues undetected by other methods, and identifying a bug in a family of Intel processors.

EPFL2017

i-PI 2.0: A universal force engine for advanced molecular simulations

Sébastien Pierre Bienvenue, Michele Ceriotti, Bingqing Cheng, Alice Cuzzocrea, Benjamin Aaron Helfrecht, Przemyslaw Juda, Venkat Kapil, Robert Horst Meissner, Riccardo Petraglia, Mariana Rossi Carvalho, Gareth Aneurin Tribello, David Mark Wilkins

Progress in the atomic-scale modeling of matter over the past decade has been tremendous. This progress has been brought about by improvements in methods for evaluating interatomic forces that work by either solving the electronic structure problem explicitly, or by computing accurate approximations of the solution and by the development of techniques that use the Born-Oppenheimer (BO) forces to move the atoms on the BO potential energy surface. As a consequence of these developments it is now possible to identify stable or metastable states, to sample configurations consistent with the appropriate thermodynamic ensemble, and to estimate the kinetics of reactions and phase transitions. All too often, however, progress is slowed down by the bottleneck associated with implementing new optimization algorithms and/or sampling techniques into the many existing electronic-structure and empirical-potential codes. To address this problem, we are thus releasing a new version of the i-PI software. This piece of software is an easily extensible framework for implementing advanced atomistic simulation techniques using interatomic potentials and forces calculated by an external driver code. While the original version of the code (Ceriotti et al., 2014) was developed with a focus on path integral molecular dynamics techniques, this second release of i-PI not only includes several new advanced path integral methods, but also offers other classes of algorithms. In other words, i-PI is moving towards becoming a universal force engine that is both modular and tightly coupled to the driver codes that evaluate the potential energy surface and its derivatives. Program summary Program Title: i-PI Program Files doi: http://dx.doLorg/10.17632/x792grbm9g.1 Licensing provisions: GPLv3, MIT Programming language: Python External routines/libraries: NumPy Nature of problem: Lowering the implementation barrier to bring state-of-the-art sampling and atomistic modeling techniques to ab initio and empirical potentials programs. Solution method: Advanced sampling methods, including path-integral molecular dynamics techniques, are implemented in a Python interface. Any electronic structure code can be patched to receive the atomic coordinates from the Python interface, and to return the forces and energy that are used to integrate the equations of motion, optimize atomic geometries, etc. Restrictions: This code does not compute interatomic potentials, although the distribution includes sample driver codes that can be used to test different techniques using a few simple model force fields. (C) 2018 Elsevier B.V. All rights reserved.

ELSEVIER SCIENCE BV2019

i-PI 2.0: A universal force engine for advanced molecular simulations

ELSEVIER SCIENCE BV2019

Easy and Accurate Hardware-based Program Performance Monitoring

Andrzej Pawel Nowak

EPFL2017