Linebacker: Preserving Victim Cache Lines in Idle Register Files of GPUs

Modern GPUs suffer from cache contention due to the limited cache size that is shared across tens of concurrently running warps. To increase the per-warp cache size prior techniques proposed warp throttling which limits the number of active warps. Warp throttling leaves several registers to be dynamically unused whenever a warp is throttled. Given the stringent cache size limitation in GPUs this work proposes a new cache management technique named Linebacker (LB) that improves GPU performance by utilizing idle register file space as victim cache space. Whenever a CTA becomes inactive, linebacker backs up the registers of the throttled CTA to the off-chip memory. Then, linebacker utilizes the corresponding register file space as victim cache space. If any load instruction finds data in the victim cache line, the data is directly copied to the destination register through a simple register-register move operation. To further improve the efficiency of victim cache linebacker allocates victim cache space only to a select few load instructions that exhibit high data locality. Through a careful design of victim cache indexing and management scheme linebacker provides 29.0% of speedup compared to the previously proposed warp throttling techniques.

Processeur à haut degré de parallélisme basé sur des composantes sérielles

Ralph Hoffmann

Nowaday, the world of processors is still dominated by the RISC architectures, which foundations have been laid down in the 70's. The RISC concept may be summarized by one word : simplicity. With this concept, much simpler architectures are born, in particular from their instruction sets point of view. As a result, these architectures have a more efficient instruction decoding and smaller computing units which in turn provides the ability to have more of these units for the same price, a reduced clock per instruction ratio — a single cycle per instruction ideally — and increased clock frequencies. As time goes by, RISC architectures have shown outstanding performances, on one hand with the spectacular improvements of semiconductor technologies, and on the other hand with the use of large cache memories and with the development of new ways of execution such as speculative execution. Later developments have greatly altered the original spirit of the simplicity of the RISC concept. Current high performance processors are extremely complex and difficult to design. Moreover, semiconductor technologies begin to show serious problems ignored or neglected for a long time. In this thesis, we would like to introduce a new architecture. Two complementary aspects have been taken in consideration : the design of the processor's units (arithmetic operators, control logic, and so on) and the architectural model, the way the processing units are used. The first aspect has been addressed with an unusual and original approach which involves the use of the serial arithmetic principles. Serial arithmetic tends to reduce the hardware cost, enables higher clock rates, and shows interesting properties which will be developped later on. The research of an architectural model is driven by the same wish of simplicity, recurrent in this field, and will inevitably shift the complexity from the processor to the compiler. This simplification relies on two things : first, the way we think of the problem is inspired by the EPIC philosophy (Explicitly Parallel Instruction Computer), the latest offspring of the VLIW model which advocates the delegation of decisions from the hardware to the software. In one sentence, "the compiler decides, the hardware executes". Second, the way we act has been given by the TTA model (Transport Triggered Architecture), the last offspring of the MOVE model which not only removes from the processor the responsability to decide which processing unit to be used for each instruction (it's already the case in a VLIW processor), but also releases the processor from scheduling the data flows. In doing so, the processor is confined to provide a set of computing units, storage units and means of communication, and the compiler decides how to move the data. In this way, a part of the control logic is removed, communication resources may be fine-tuned and the compiler's code-optimization can be enhanced. With the combination of those elements, we wish to get an architecture which exhibits a better computing power/consumption ratio, which is more flexible, easier to developp and to evolve.

EPFL2004

A Primer on Hardware Prefetching

Babak Falsafi

Since the 1970’s, microprocessor-based digital platforms have been riding Moore’s law, allowing for doubling of density for the same area roughly every two years. However, whereas microprocessor fabrication has focused on increasing instruction execution rate, memory fabrication technologies have focused primarily on an increase in capacity with negligible increase in speed. This divergent trend in performance between the processors and memory has led to a phenomenon referred to as the “Memory Wall.” To overcome the memory wall, designers have resorted to a hierarchy of cache memory levels, which rely on the principal of memory access locality to reduce the observed memory access time and the performance gap between processors and memory. Unfortunately, important workload classes exhibit adverse memory access patterns that baffle the simple policies built into modern cache hierarchies to move instructions and data across cache levels. As such, processors often spend much time idling upon a demand fetch of memory blocks that miss in higher cache levels. Prefetching—predicting future memory accesses and issuing requests for the corresponding memory blocks in advance of explicit accesses—is an effective approach to hide memory access latency. There have been a myriad of proposed prefetching techniques, and nearly every modern processor includes some hardware prefetching mechanisms targeting simple and regular memory access patterns. This primer offers an overview of the various classes of hardware prefetchers for instructions and data proposed in the research literature, and presents examples of techniques in- corporated into modern microprocessors.

Morgan & Claypool2014

Processeur à haut degré de parallélisme basé sur des composantes sérielles

Ralph Hoffmann

EPFL2004