Designing a Virtual Runtime for FPGA Accelerators in the Cloud

FPGAs can provide high performance and energy efficiency to many applications; therefore, they are attractive computing platforms in a cloud environment. However, FPGA application development requires extensive hardware design knowledge which significantly limits the potential user base. Moreover, in a cloud setting, allocating a whole FPGA to a user is often wasteful and not cost effective due to low device utilization. To make FPGA application development easier, firstly, we propose a methodology that provides clean abstractions with high-level APIs and a simple execution model that supports both software and hardware execution. Secondly, to improve device utilization and share the FPGA among multiple users, we developed a lightweight runtime system that provides hardware-assisted memory virtualization and memory protection, enabling multiple applications to simultaneously execute on the device.

Virtualized Execution Runtime for FPGA Accelerators in the Cloud

Mikhail Asiatici, Nithin George, Paolo Ienne

FPGAs offer high performance coupled with energy efficiency making them extremely attractive computational resources within a cloud ecosystem. However, to achieve this integration and make them easy to program, we first need to enable users with varying expertise to easily develop cloud applications that leverage FPGAs. With the growing size of FPGAs, allocating them monolithically to users can be wasteful due to potentially low device utilization. Hence, we also need to be able to dynamically share FPGAs among multiple users. To address these concerns, we propose a methodology and a runtime system that together simplify the FPGA application development process by providing: 1) a clean abstraction with high-level APIs for easy application development; 2) a simple execution model that supports both hardware and software execution; and 3) a shared memory-model which is convenient to use for the programmers. Akin to an operating system on a computer, our lightweight runtime system enables the simultaneous execution of multiple applications by virtualizing computational resources, i.e., FPGA resources and on-board memory, and offers protection facilities to isolate applications from each other. In this paper, we illustrate how these features can be developed in a lightweight manner and quantitatively evaluate the performance overhead they introduce on a small set of applications running on our proof of concept prototype. Our results demonstrate that these features only introduce marginal performance overheads. More importantly, by sharing resources for simultaneous execution of multiple user applications, our platform improves FPGA utilization and delivers higher aggregate throughput compared to accessing the device in a time-shared manner.

2017

Scaling Out Bioinformatics in the Data Center

Sam David Whitlock

Whole Genome Sequencing is a process in the field of bioinformatics that transforms biological samples of DNA into an electronic dataset of genetic bases. The process consists of two sequential components. First, the laboratory process of sequencing transforms the biological DNA samples into a digital format by transcribing the sequence of bases that make up short snippets of DNA. Second, a genomic workflow uses software to transform this data into a representation that is useful for genomic analysis. Recent advancements in High-Throughput Sequencing technology enable the laboratory phase to produce data faster and at a lower cost than prior techniques were capable of. The applications and file formats used in workflows have not undergone commensurate technological advancement in order to accommodate this deluge of genomic data. This thesis introduces a redesign of genomic workflows, their component applications, and the underlying file formats in order to scale out workflows across a data center. The design builds upon on two design components. First, a unified file format supplants the myriad existing formats in order to accommodate scale-out multi-machine I/O. The file format imposes minimal feature requirements upon the storage system, thereby enabling its use in high- performance systems for processing and cost-effective cold storage systems for long-term storage. Second, a new cloud computing framework provides an API for composing workflows in an abstract logical description and delegating the execution of the logic to a common runtime. The framework's runtime executes the logical workflow description on scale-out hardware resources while abstracting the execution details. We combine the file format and framework to build a new set of workflows that scale out across data center resources. These scale-out workflows incorporate existing workflow applications (compartmentalized into libraries that the framework invokes) and new applications that leverage the features provided by the scale-out architecture. All workflows delegate work distribution and task concurrency to the framework's runtime and utilize a common set of subcomponents for auxiliary code (e.g., I/O with various storage systems, processing different file formats). These workflows are able to scale out across a data center to the point of saturating the throughput of one or more hardware resources.

EPFL2019

Optimizing network performance in virtual machines

Aravind Menon

In recent years, there has been a rapid growth in the adoption of virtual machine technology in data centers and cluster environments. This trend towards server virtualization is driven by two main factors: the savings in hardware cost that can be achieved through the use of virtualization, and the increased flexibility in the management of hardware resources in a cluster environment. An important consequence of server virtualization is the negative impact it has on the networking performance of server applications running in virtual machines (VMs). In this thesis, we address the problem of efficiently virtualizing the network interface in Type-II virtual machine monitors. In the Type-II architecture, the VMM relies on a special, 'host' operating system to provide the device drivers to access I/O devices, and executes the drivers within the host operating system. Using the Xen VMM as an example of this architecture, we identify fundamental performance bottlenecks in the network virtualization architecture of Type-II VMMs. We show that locating the device drivers in a separate host VM is the primary reason for performance degradation in Type-II VMMs, because of two reasons: a) the switching between the guest and the host VM for device driver invocation, and, b) I/O virtualization operations required to transfer packets between the guest and the host address spaces. We present a detailed analysis of the virtualization overheads in the Type-II I/O architecture, and we present three solutions which explore the performance that can be achieved while performing network virtualization at three different levels: in the host OS, in the VMM, and in the NIC hardware. Our first solution consists of a set of packet aggregation optimizations that explores the performance achievable while retaining the Type-II I/O architecture in the Xen VMM. This solution retains the core functionality of I/O virtualization, including device driver execution, in the Xen 'driver domain'. With this set of optimizations, we achieve an improvement by a factor of two to four in the networking performance of Xen guest domains. In our second solution, we move the task of I/O virtualization and device driver execution from the host OS to the Xen hypervisor. We propose a new I/O virtualization architecture, called the TwinDrivers framework, which combines the performance advantages of Type-I VMMs with the safety and software engineering benefits of Type-II VMMs. (In a Type-I VMM, the device driver executes directly in the hypervisor, and gives much better performance than a Type-II VMM). The TwinDrivers architecture results in another factor of two improvements in networking performance for Xen guest domains. Finally, in our third solution, we describe a hardware based approach to network virtualization, in which we move the task of network virtualization into the network interface card (NIC). We develop a specialized network interface (CDNA) which allows guest operating systems running in VMs to directly access a private, virtual context on the NIC for network I/O, bypassing the host OS entirely. This approach also yields performance benefits similar to the TwinDrivers software-only approach. Overall, our solutions help significantly bridge the gap between the network performance in a virtualized environment and a native environment, eventually achieving network performance in a virtual machine within 70% of the native performance.