Memory hierarchy

Related publications (43)

Dynamic Resource Allocation for Database Servers Running on Virtual Storage

Jin Chen, Daniel Lupei, Adrian Daniel Popescu

We introduce a novel multi-resource allocator to dynamically allocate resources for database servers running on virtual storage. Multi-resource allocation involves proportioning the database and storage server caches, and the storage bandwidth between applications according to overall performance goals. The problem is challenging due to the interplay between different resources, e.g., changing any cache quota affects the access pattern at the cache/disk levels below it in the storage hierarchy. We use a combination of on-line modeling and sampling to arrive at near-optimal configurations within minutes. The key idea is to incorporate access tracking and known resource dependencies e.g., due to cache replacement policies, into our performance model. In our experimental evaluation, we use both microbenchmarks and the industry standard benchmarks TPC-W and TPC-C. We show that our multi-resource allocation approach improves application performance by up to factors of 2.9 and 2.4 compared to state-of-the-art single-resource controllers, and their ad-hoc combination, respectively.

2009

Adding flexibility to multi-tenant networks

Georgios Ioannidis

Cloud computing has been experiencing sharp development over the last years, leading to an increased demand for application migration to the cloud. Cloud providers, in an effort to attract more customers and earn their confidence, offer to tenants the illusion of an isolated network, exposing familiar abstractions. At the same time, creating this illusion poses challenging problems for the providers, as one tenant's traffic may interfere with another's in complicated, unpredictable ways. First, new challenges have arisen in administering access-control rules (ACLs). On the one hand, installing ACLs at the server is incompatible with bare-metal support and introduces unnecessary performance overhead. On the other hand, offloading the most popular ACLs on the limited hardware memory in Top-of-Rack (ToR) switches should not be conducted naïvely, as the existence of wildcard rules presents inter-rule dependencies that must be respected. Second, tenants' demands have evolved beyond requesting hardware resources; for instance, tenants may require bandwidth provisions between their resources or optimized access to a specific cloud service, e.g., a Mail server or a Database. Cloud providers have not adequately adapted to these expanding demands, therefore elevating hardware resources to "first class citizens," as non-hardware constraints are not considered during resource allocation, instead they are applied afterwards. In this thesis we propose two architectures that facilitate cloud providers in managing their shared network resources in a flexible way. First, we demonstrate virtual flow tables, a ToR architecture that handles ACLs using a two-level memory hierarchy. The most popular ACLs are stored in the limited hardware memory, respecting any dependencies between wildcard rules, while the ToR's supervisor engine maintains access to the entire ACL rule-set. Second, we present a two-tiered architecture for scheduling cloud resources, consisting of a resource-agnostic scheduling layer and a resource-specific enforcement layer. Network resources and constraints are taken into consideration during resource scheduling, instead of afterwards, while resource provisioning, as well as general network-management policies, are delegated to the resource-specific tier.

EPFL2018

Exploiting Compute Caches for Memory Bound Vector Operations

Gabriel Falcao Paiva Fernandes, Paolo Ienne, João Miguel Morgado Pereira Vieira

To reduce the average memory access time, most current processors make use of a multilevel cache subsystem. However, despite the proven benefits of such cache structures in the resulting throughput, conventional operations such as copy, simple maps and reductions still require moving large amounts of data to the processing cores. This imposes significant energy and performance overheads, with most of the execution time being spent moving data across the memory hierarchy. To mitigate this problem, a Cache Compute System (CCS) that targets memory-bound kernels such as map and reduce operations is proposed. The developed CCS takes advantage of long cache lines and data locality to avoid data transfers to the processor and exploits the intrinsic parallelism of vector compute units to accelerate a set of 48 operations commonly used in map and reduce patterns. The CCS was validated by integrating it with an MB-Lite soft-core in a Xilinx Virtex-7 VC709 Development Board. When compared to the MB-Lite core, the proposed CCS presents performance improvements in the execution of the commands ranging from 4x to 408x, and energy efficiency gains from 6x to 328x.

IEEE2018

RRAM Crossbar Arrays for Storage Class Memory Applications : Throughput and Density Considerations

Alexandre Sébastien Julien Levisse

As more and more high density memories are required to satisfy the Internet of Things ecosystem, academics and industrials are looking for an intermediate solution to fill the gap between DRAM and Flash NAND in the memory hierarchy. The emergence of Resistive Switching Technologies (RRAM) proposes a potential solution to this demand for fast, low cost, high density and non-volatile memory. However, nowadays transistor less RRAM-based architectures, such as Crosspoint, suffers of several issues such as sneakpath, IRdrop and periphery overhead. In this work, we propose to explore the positioning of RRAM crosspoint memories regarding DRAM and NAND in terms of density and write throughput. We present several design guidelines then show that for the optimal RRAM crosspoint architecture (2-layers with common bitline), massively multiple bank write is the solution to optimize density and write throughput to around 20-100Gbit/cm2 and 200-500MB/s respectively for 32 to 64 parallel access.

IEEE2018

RRAM Crossbar Arrays for Storage Class Memory Applications : Throughput and Density Considerations

Alexandre Sébastien Julien Levisse

As more and more high density memories are required to satisfy the Internet of Things ecosystem, academics and industrials are looking for an intermediate solution to fill the gap between DRAM and Flash NAND in the memory hierarchy. The emergence of Resistive Switching Technologies (RRAM) proposes a potential solution to this demand for fast, low cost, high density and non-volatile memory. However, nowadays transistor-less RRAM-based architectures, such as Crosspoint, suffers of several issues such as sneakpath, IRdrop and periphery overhead. In this work, we propose to explore the positioning of RRAM crosspoint memories regarding DRAM and NAND in terms of density and write throughput. We present several design guidelines then show that for the optimal RRAM crosspoint architecture (2-layers with common bitline), massively multiple bank write is the solution to optimize density and write throughput to around 20-100Gbit/cm2 and 200-500MB/s respectively for 32 to 64 parallel access.

2018

Rebooting Virtual Memory with Midgard

Atri Bhattacharyya, Babak Falsafi, Siddharth Gupta, Yunho Oh, Mathias Josef Payer

Computer systems designers are building cache hierarchies with higher capacity to capture the ever-increasing working sets of modern workloads. Cache hierarchies with higher capacity improve system performance but shift the performance bottleneck to address translation. We propose Midgard, an intermediate address space between the virtual and the physical address spaces, to mitigate address translation overheads without program-level changes. Midgard leverages the operating system concept of virtual memory areas (VMAs) to realize a single Midgard address space where VMAs of all processes can be uniquely mapped. The Midgard address space serves as the namespace for all data in a coherence domain and the cache hierarchy. Because real-world workloads use far fewer VMAs than pages to represent their virtual address space, virtual to Midgard translation is achieved with hardware structures that are much smaller than TLB hierarchies. Costlier Midgard to physical address translations are needed only on LLC misses, which become much less frequent with larger caches. As a consequence, Midgard shows that instead of amplifying address translation overheads, memory hierarchies with large caches can reduce address translation overheads. Our evaluation shows that Midgard achieves only 5% higher address translation overhead as compared to traditional TLB hierarchies for 4KB pages when using a 16MB aggregate LLC. Midgard also breaks even with traditional TLB hierarchies for 2MB pages when using a 256MB aggregate LLC. For cache hierarchies with higher capacity, Midgard's address translation overhead drops to near zero as secondary and tertiary data working sets fit in the LLC, while traditional TLBs suffer even higher degrees of address translation overhead.

2021

, ,

Datacenter operators have started deploying Persistent Memory (PM), leveraging its combination of fast access and persistence for significant performance gains. A key challenge for PM-aware software is to maintain high performance while achieving atomic durability. The latter typically requires the use of logging, which introduces considerable overhead with additional CPU cycles, write traffic, and ordering requirements. In this paper, we exploit the data multiversioning inherent in the memory hierarchy to achieve atomic durability without logging. Our design, LAD, relies on persistent buffering space at the memory controllers (MCs)—already present in modern CPUs—to speculatively accumulate all of a transaction’s updates before they are all atomically committed to PM. LAD employs an on-chip distributed commit protocol in hardware to manage the distributed speculative state each transaction accumulates across multiple MCs. We demonstrate that LAD is a practical design relying on modest hardware modifications to provide atomically durable transactions, while delivering up to 80% of ideal—i.e., PM-oblivious software’s—performance.

2019

Utilization of edge devices has exploded in the last decade, with such use cases as wearable devices, autonomous driving, and smart homes. As their ubiquity grows, so do expectations of their capabilities. Simultaneously, their formfactor and use cases limit power availability. Thus, improving performance while limiting area and power consumption is paramount. In this vein, in-SRAM Computing (iSC) moves computation from the CPU into the SRAM memory hierarchy. This has multiple benefits. First, reduced data movement mitigates power consumption and latency. Second, the entire memory array can be utilized to perform hundreds of concurrent operations. This thesis exploits iSC while addressing the aforementioned challenges via a BitLine Accelerator for Devices on the Edge (BLADE). BLADE can be implemented in any SRAM system and utilizes local wordline groups to perform computations at a frequency 2.8x higher than state-of-the-art iSC architectures. BLADE is thoroughly simulated, fabricated, and benchmarked at the transistor, architecture, and software abstraction levels. Experimental results demonstrate performance/energy gains over an equivalent NEON accelerated processor for a variety of edge device workloads, namely, cryptography (4x performance gain/6x energy reduction), video encoding (6x/2x), and convolutional neural networks (3x/1.5x), while maintaining the highest frequency/energy ratio (up to 2.2Ghz@1V) of any conventional iSC computing architecture, and a low area overhead of less than 8%. With BLADE implemented, the possibilities for enhancement are manifold, with one such example being approximate computing. To this end, a CArryless Partial Product InExact Multiplier (CAPPIEM) halves multiplication latency while incurring negligible area overhead. As a standalone multiplier, CAPPIEM reduces the area/power-delay-product by 73/43%, respectively. Further, CAPPIEM has the unique property of computing exact results when one input is a Fibonacci encoded value. This property is exploited via a retraining strategy which quantizes neural network weights to Fibonacci values, ensuring exact computation during inference. Benchmarking on Squeezenet 1.0, DenseNet-121, and ResNet-18 demonstrate accuracy degradations of only 0.4/1.1/1.7%, while improving training time by up to 300x. A second BLADE enhancement is the use of Hybrid Caches (HCs) consisting of both SRAM and eNVRAM bitcells. HCs increase capacity and power savings via eNVRAM's small area footprint and low leakage energy. However, eNVRAMs also incur long write latency and limited endurance. To mitigate these drawbacks, this thesis presents SHyCache, an HC architecture and supporting programming model. By explicitly allocating variables with high read/write access ratios to the eNVRAM array, SHyCache reduces access time, power consumption, and area overhead, while maintaining maximal utilization efficiency and ease of programming. Benchmarks on a range of cache hierarchy variations using three deep neural networks demonstrate a design space that can be exploited to optimize performance, power consumption, or endurance, while demonstrating maximum performance gains of 1.7/1.4/1.3x and power consumption reductions of 5.1/5.2/5.4x.

EPFL2022

Automatic Synthesis of Out-of-Core Algorithms

Ioannis Klonatos, Christoph Koch, Viktor Kuncak, Andres Nötzli, Andrej Spielmann

We present a system for the automatic synthesis of efficient algorithms specialized for a particular memory hierarchy and a set of storage devices. The developer provides two independent inputs: 1) an algorithm that ignores memory hierarchy and external storage aspects; and 2) a description of the target memory hierarchy, including its topology and parameters. Our system is able to automatically synthesize memory-hierarchy and storage-device-aware algorithms out of those specifications, for tasks such as joins and sorting. The framework is extensible and allows developers to quickly synthesize custom out-of-core algorithms as new storage technologies become available.

ACM Press2013

, , , ,

Large-scale online services are commonly structured as a network of software tiers, which communicate over the datacenter network using RPCs. Ongoing trends towards software decomposition have led to the prevalence of tiers receiving and generating RPCs with runtimes of only a few microseconds. With such small software runtimes, even the smallest latency overheads in RPC handling have a significant relative performance impact. In particular, we find that growing network bandwidth introduces queuing effects within a server’s memory hierarchy, considerably hurting the response latency of fine-grained RPCs. In this work we introduce NeBuLa, an architecture optimized to accelerate the most challenging microsecond-scale RPCs, by leveraging two novel mechanisms to drastically improve server throughput under strict tail latency goals. First, NeBuLa reduces detrimental queuing at the memory controllers via hardware support for efficient in-LLC network buffer management. Second, NeBuLa’s network interface steers incoming RPCs into the CPU cores’ L1 caches, improving RPC startup latency. Our evaluation shows that NeBuLa boosts the throughput of a state-of-the-art key- value store by 1.25–2.19x compared to existing proposals, while maintaining strict tail latency goals.

2020