Reflexes: Abstractions for Highly Responsive Systems

Commercial Java virtual machines are designed to maximize the performance of applications at the expense of predictability. High throughput garbage collection algorithms, for example, can introduce pauses of 100 milliseconds or more. We are interested in supporting applications with response times in the tens of microseconds and their integration with larger timing-oblivious applications in the same Java virtual machine. We propose Reflexes, a new abstraction for writing highly responsive systems in Java and investigate the virtual machine support needed to add Reflexes to a Java environment. Our implementation of Reflexes was evaluated on several programs including an audio-processing application running at 22.05KHz. The number of missed deadlines, less than 0.2% for 10 million observations, compares favorably to a native C implementation.

Architecture Exploration and Optimization of Heterogeneous Many-Core Compute and Memory Architectures with Architectural Extensions

Yasir Mahmood Qureshi

The expeditious proliferation of Internet connectivity and the growing adoption of digital products have transformed various spheres of our everyday lives. This increased digitization of society has led to the emergence of new applications, which are deployed all the way from High Performance Computing (HPC) systems and cloud servers to mobile devices. These new emerging applications like video analytics, autonomous driving, natural language processing, content recommendation, bioinformatics, and genome sequencing, have different performance/energy requirements, and are deployed on a variety of different platforms. To meet the performance and Quality-of-Service (QoS)constraints, cloud servers, and HPC platforms are comprised of many-core processors. Even the processors in today's mobile and edge devices are multi-core. To optimize energy efficiency and performance, a system-level simulator is required. This simulator must be capable of simultaneously executing multi-threaded applications in a full-system environment with a complete operating system (OS), on a heterogeneous many-core system.In this thesis, I present gem5-X, a system-level simulation platform to optimize many-core heterogeneous compute and memory architectures. Gem5-X exploration and optimization methodology is also proposed to optimize both the compute and memory sub-system for multi-thread applications. Gem5-X extends gem5 with architectural extensions for the compute and memory sub-systems, including in-caching computing accelerator, clustered heterogeneous cores, in-memory computing engine, and 3D stacked High Bandwidth Memory v2 (HBM2). It also adds support enhancements to gem5, including, Virtual Machine (VM) support, profiling support in the simulator using gperf profiler, enhanced checkpointing, and file sharing between the simulated and the host system. Finally, all the extensions and enhancements are fully supported in Full System (FS)mode of gem5-X, enabling booting and running a full Linux stack on top of the simulator, allowing almost any application running in a Linux system to be executed using gem5-X.To demonstrate the capabilities of gem5-X, various compute-dominated and memory-dominated applications are used as case studies. These state-of-the-art applications include video encoding, video analytics, VMs in the cloud, Binary Neural Networks (BNNs) and Recurrent Neural Networks (RNNs) as compute-intensive workloads. In addition to compute-dominated workloads, memory-dominated applications, including genome sequence alignment based on Next Generation Sequencing (NGS) techniques, and Convolutional Neural Networks (CNNs) are presented as case studies to demonstrate the memory sub-system extensions of gem5-X. Gem5-X exploration methodology is used to optimize architectures for these applications, achieving both performance and energy benefits. All the applications and workloads used in this thesis are case studies to showcase the capabilities of gem5-X. Gem5-X is generic and can be used to optimize and explore architectures for any multi-threaded (or single-threaded) application that can run on a Linux-based OS.

EPFL2021

Continuations in the Java Virtual Machine

Antonio Cunei, Iulian Dragos

Continuations have received considerable attention lately as a possible solution to web application development. Other uses proposed in the past, such as cooperative threading, coroutines or writing iterators, made them an attractive feature of dynamically typed languages. We present issues involved in adding continuations to a statically typed, object-oriented language like Java, and its implementation in the Java Virtual Machine. We propose three different flavors of continuations, and study their interactions with the base language, focusing on Java's concurrency model. We describe our implementation in Ovm, a realtime Java Virtual Machine, and discuss open issues.

Technische Universität Berlin2007

Reflexes: programming abstractions for highly responsive computing in Java

Jesper Honig Spring

Achieving sub-millisecond response times in a managed language environment such as Java introduces significant implementation challenges. The Achilles' heel of Java is its reliance on automatic memory management for reclaiming dead objects. Typically, the garbage collectors used in commercial Java virtual machines are designed to maximize the performance of applications at the expense of predictability. Moreover, it is non-deterministic as to when and for how long the garbage collector will run. As a consequence garbage collection introduces execution interference that can easily reach hundreds of milliseconds, preventing the timeliness requirements of the real-time systems from being satisfied. Another source of interference relates to the integration of real-time tasks with a time-oblivious application. Typical programming practices for sharing data between threads involve synchronized access to some shared data structure through mutual exclusion. In a real-time system this might lead to unbounded blocking of the real-time thread, so-called priority inversion, causing serious deadlines infringements. Faced with these challenges a system designer typically has two options: to deploy a realtime garbage collector (RTGC), or to restrict the programming model. RTGCs achieve much improved predictability over traditional stop-the-world garbage collectors by interleaving the application execution with the garbage collection. Recent advances in real-time garbage collection algorithms have reduced latency to approximately 1 millisecond. Nevertheless, some applications, e.g., safety critical applications certified according to DO-178B, have temporal requirements beyond what is possible with state-of-the-art RTGCs. Moreover, real-time garbage collectors still face the problem of priority inversion. This thesis presents Reflexes, a simple, statically type-safe restricted programming model facilitating the construction of highly-responsive applications in Java. Reflexes are designed to make it easy to write and integrate simple periodic tasks or complex stream processors, both observing real-time timing constraints in the sub-millisecond range, into larger time-oblivious Java applications. Reflex tasks run in a part of memory free from garbage collection interference, and maintain a class-based separation between object lifetimes, enabling reclamation of periodic garbage in constant time. Tasks are organized in a graph and communicate through uni-directional, non-blocking channels. Furthermore, Reflexes enable non-blocking interaction between real-time tasks and time-oblivious code using methods with transactional semantics, circumventing typical problems of priority inversion when using common programming practices for synchronizing access to shared data. Reflexes specify a set of static safety checks preventing dangling pointers and preventing a Reflex from observing heap-allocated objects in an inconsistent state as they are, e.g., when being copied by a garbage collector. These checks are enforced statically by an extension of the standard Java compiler to guarantee correctness. We describe two implementations of Reflexes: a stand-alone prototype implementation built on top of a research Java real-time virtual machine, and an implementation based on an integration of Reflexes with two existing restricted programming models into a single unified framework, Flexotask, running on top of an industrial-strength Java real-time virtual machine. Whereas the prototype implementation of Reflexes is limited to a virtual machine with uni-processor support and exploits non-standard features of the virtual machine, the latter implementation leverages the multi-processor support of an industrial-strength virtual machine with minimal extensions. For both implementations we report separately on a number of encouraging results from empirical experiments using benchmark and real-world applications. Specifically, our experiments show that Reflexes in both cases are capable of achieving sub-millisecond response time with a high degree of predictability.