Automatic parallelization | EPFL Graph Search

Automatic parallelization, also auto parallelization, or autoparallelization refers to converting sequential code into multi-threaded and/or vectorized code in order to use multiple processors simultaneously in a shared-memory multiprocessor (SMP) machine. Fully automatic parallelization of sequential programs is a challenge because it requires complex program analysis and the best approach may depend upon parameter values that are not known at compilation time. The programming control structures on which autoparallelization places the most focus are loops, because, in general, most of the execution time of a program takes place inside some form of loop. There are two main approaches to parallelization of loops: pipelined multi-threading and cyclic multi-threading. For example, consider a loop that on each iteration applies a hundred operations, and runs for a thousand iterations. This can be thought of as a grid of 100 columns by 1000 rows, a total of 100,000 operations. Cyclic multi-threading assigns each row to a different thread. Pipelined multi-threading assigns each column to a different thread. This is the first stage where the scanner will read the input source files to identify all static and extern usages. Each line in the file will be checked against pre-defined patterns to segregate into tokens. These tokens will be stored in a file which will be used later by the grammar engine. The grammar engine will check patterns of tokens that match with pre-defined rules to identify variables, loops, control statements, functions etc. in the code restart. The analyzer is used to identify sections of code that can be executed concurrently. The analyzer uses the static data information provided by the scanner-parser. The analyzer will first find all the totally independent functions and mark them as individual tasks. The analyzer then finds which tasks have dependencies. The scheduler will list all the tasks and their dependencies on each other in terms of execution and start times 6h.

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Martin Jaggi, Thijs Vogels, Hadrien Hendrikx

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Brookline2023

Machine-learning predictor of nerve fiber firing rate allows the automatic optimization of electrical stimulation protocols

Silvestro Micera, Simone Romeni, Luca Pierantoni, Gabriele Marino

Computational models have been widely employed to study the electrical stimulation of the nervous system. Still, most applications either study fundamental mechanisms underlying stimulation, or address qualitative scientific questions. When quantitative qu ...

IEEE2023