Parallelism: Programming and Performance

This lecture introduces the concepts of parallelism in programming, focusing on the trade-offs between programmability and performance. It covers the reasons for the shift towards parallel computing, such as the limitations of frequency scaling and the end of the 'free lunch'. The lecture discusses the parallelization of code using various programming abstractions like OpenMP and pthreads, emphasizing the importance of expressing parallelism in code. It also explores the execution models of simple and multicore CPUs, as well as GPUs, highlighting the principles of parallel computing and the implications of Amdahl's Law. The lecture concludes with an overview of shared memory parallel programming using OpenMP, synchronization techniques, and the importance of load balancing and communication models.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (52)

Related concepts (180)

CS-307: Introduction to multiprocessor architecture

Multiprocessors are a core component in all types of computing infrastructure, from phones to datacenters. This course will build on the prerequisites of processor design and concurrency to introduce

Simultaneous multithreading

Simultaneous multithreading (SMT) is a technique for improving the overall efficiency of superscalar CPUs with hardware multithreading. SMT permits multiple independent threads of execution to better use the resources provided by modern processor architectures. The term multithreading is ambiguous, because not only can multiple threads be executed simultaneously on one CPU core, but also multiple tasks (with different page tables, different task state segments, different protection rings, different I/O permissions, etc.

Multi-core processor

A multi-core processor is a microprocessor on a single integrated circuit with two or more separate processing units, called cores, each of which reads and executes program instructions. The instructions are ordinary CPU instructions (such as add, move data, and branch) but the single processor can run instructions on separate cores at the same time, increasing overall speed for programs that support multithreading or other parallel computing techniques.

Dynamic frequency scaling

Dynamic frequency scaling (also known as CPU throttling) is a power management technique in computer architecture whereby the frequency of a microprocessor can be automatically adjusted "on the fly" depending on the actual needs, to conserve power and reduce the amount of heat generated by the chip. Dynamic frequency scaling helps preserve battery on mobile devices and decrease cooling cost and noise on quiet computing settings, or can be useful as a security measure for overheated systems (e.g.

Data parallelism

Data parallelism is parallelization across multiple processors in parallel computing environments. It focuses on distributing the data across different nodes, which operate on the data in parallel. It can be applied on regular data structures like arrays and matrices by working on each element in parallel. It contrasts to task parallelism as another form of parallelism. A data parallel job on an array of n elements can be divided equally among all the processors.

Von Neumann architecture

The von Neumann architecture—also known as the von Neumann model or Princeton architecture—is a computer architecture based on a 1945 description by John von Neumann, and by others, in the First Draft of a Report on the EDVAC. The document describes a design architecture for an electronic digital computer with these components: A processing unit with both an arithmetic logic unit and processor registers A control unit that includes an instruction register and a program counter Memory that stores data and instructions External mass storage Input and output mechanisms The term "von Neumann architecture" has evolved to refer to any stored-program computer in which an instruction fetch and a data operation cannot occur at the same time (since they share a common bus).

Parallel Computing: Principles and OpenMP CS-307: Introduction to multiprocessor architecture

Covers the principles of parallel computing and introduces OpenMP for creating concurrent code from serial code.

Principles of Parallel Computing: OpenMP CS-307: Introduction to multiprocessor architecture

Explores the principles of parallel computing, focusing on OpenMP as a tool for creating concurrent code from serial code.

Parallel Programming I CS-471: Advanced multiprocessor architecture

Covers the basics of parallel programming, including concurrency, forms of parallelism, synchronization, and programming models like PThreads and OpenMP.

Principles of Parallelism

Covers the basics of parallelism, including physical examples, historical context, multicore era, and parallel collections in Scala.

Neuromorphic Computing: Challenges and Future BIOENG-450: In silico neuroscience

Discusses the challenges and future of neuromorphic computing, comparing digital computers and specialized hardware, such as SpiNNaker and NEST, while exploring the Human Brain Project's Neuromorphic Computing Platform.