Asynchronous Branch-Paralle Simulation of Detailed Neuron Models

Simulations of electrical activity of networks of morphologically detailed neuron models allow for a better understanding of the brain. State-of-the-art simulations describe the dynamics of ionic currents and biochemical processes within branching topological representations of the neurons. Acceleration of such simulation is possible in the weak scaling limit by modeling neurons as indivisible computation units and increasing the computing power. Strong scaling and simulations close to biological time are difficult, yet required, for the study of synaptic plasticity and other use cases requiring simulation of neurons for long periods of time. Current methods rely on parallel Gaussian Elimination, computing triangulation and substitution of many branches simultaneously. Existing limitations are: (a) high heterogeneity of compute time per neuron leads to high computational load imbalance; and (b) difficulty in providing a computation model that fully utilizes the computing resources on distributed multi-core architectures with Single Instruction Multiple Data (SIMD) capabilities. To address these issues, we present a strategy that extracts flow-dependencies between parameters of the ODEs and the algebraic solver of individual neurons. Based on the resulting map of dependencies, we provide three techniques for memory, communication, and computation reorganization that yield a bad-balanced distributed asynchronous execution. The new computation model distributes datasets and balances computational workload across a distributed memory space, exposing a tree-based parallelism of neuron topological structure, an embarrassingly parallel execution model of neuron subtrees, and a SIMD acceleration of subtree state updates. The capabilities of our methods are demonstrated on a prototype implementation developed on the core compute kernel of the NEURON scientific application, built on the HPX runtime system for the ParalleX execution model. Our implementation yields an asynchronous distributed and parallel simulation that accelerates single neuron to medium-sized neural networks. Benchmark results display better strong scaling properties, finer-grained parallelism, and lower time to solution compared to the state of the art, on a wide range of distributed multi-core compute architectures.

Asynchronous Simulation of Neuronal Activity

Bruno Ricardo Da Cunha Magalhães

Simulations of the electrical activity of networks of morphologically-detailed neuron models allow for a better understanding of the brain. Short time to solution is critical in order to study long biological processes such as synaptic plasticity and learning. State-of-the-art approaches follow the Bulk Synchronous Parallel execution model based on the Message Passing Interface runtime system. The efficient simulation of networks of simple neuron models is a well researched problem. However, the efficient large-scale simulation of detailed neuron models --- including topological information and a wide range of complex biological mechanisms --- remains an open problem, mostly due to the high complexity of the model, the high heterogeneity of specifications in modern compute architectures, and the limitations of existing runtime systems. In this thesis, we explore novel methods for the asynchronous multi-scale simulation of detailed neuron models on distributed parallel compute networks. In the first study, we introduce the concept of distributed asynchronous branch-parallelism of neurons. We present a method that extracts variable dependencies across numerical resolution of neurons topological trees and the underlying algebraic solver. We demonstrate a method for the decomposition of the neuron simulation problem into interconnected resolutions of neuron subtrees. Results demonstrate a significant reduction in runtime on heterogeneous distributed, multi-core, Single Instruction Multiple Data (SIMD) computing environments. In the second study, we complement the previous effort with graph-parallelism retrieved from mathematical dependencies across the systems of Ordinary Differential Equations (ODEs) that model the activity of neurons. We describe a method for the automatic extraction of read-after-write variable dependencies, concurrent variable updates, and independent ODEs. The automation of the methods expose a computation graph of biological mechanisms interdependency, and an embarrassingly-parallel SIMD execution of mechanism instances, leading to an acceleration of the simulation. The third part of this thesis pioneers the fully-asynchronous parallel execution model, and the "exhaustive yet not speculative" stepping protocol. We apply it to our use case, and demonstrate that by removing collective synchronization of neurons in time, by utilising a memory-linear neuron representation, and by advancing neurons based on their synaptic connectivity, a substantial runtime speed-up is achievable through cache efficiency. The fourth and last part advances the aforementioned fully-asynchronous execution model with variable-order variable-timestep interpolation methods. We demonstrate low dependency of runtime and step count on volatility of solution, and a high dependency on synaptic events. We introduce a variable-step execution model that allows for non-speculative asynchronous stepping of neurons on a distributed compute network. We demonstrate higher numerical accuracy, less interpolation steps, and shorter time to solution, compared to state-of-the-art implicit fixed-step resolution. This thesis demonstrates that utilising asynchronous runtime systems is a requirement to succeed in efficiently simulating complex neuronal activity at large scale. Most contributions presented follow from first principles in numerical methods and computer science, thus providing insights for applications on a wide range of scientific domains.

EPFL2019

Continuous attractor working memory and provenance of channel models

Alexander Kevin Seeholzer

The brain is a complex biological system composed of a multitude of microscopic processes, which together give rise to computational abilities observed in everyday behavior. Neuronal modeling, consisting of models of single neurons and neuronal networks at varying levels of biological detail, can synthesize the gaps currently hard to constrain in experiments and provide mechanistic explanations of how these computations might arise. In this thesis, I present two parallel lines of research on neuronal modeling, situated at varying levels of biological detail. First, I assess the provenance of voltage-gated ion channel models in an integrative meta-analysis that investigates a backlog of nearly 50 years of published research. To cope with the ever-increasing volume of research produced in the field of neuroscience, we need to develop methods for the systematic assessment and comparison of published work. As we demonstrate, neuronal models offer the intriguing possibility of performing automated quantitative analyses across studies, by standardized simulated experiments. We developed protocols for the quantitative comparison of voltage-gated ion channels, and applied them to a large body of published models, allowing us to assess the variety and temporal development of different models for the same ion channels over the time scale of years of research. Beyond a systematic classification of the existing body of research made available in an online platform, we show that our approach extends to large-scale comparisons of ion channel models to experimental data, thereby facilitating field-wide standardization of experimentally-constrained modeling. Second, I investigate neuronal models of working memory (WM). How can cortical networks bridge the short time scales of their microscopic components, which operate on the order of milliseconds, to the behaviorally relevant time scales of seconds observed in WM experiments? I consider here a candidate model: continuous attractor networks. These can implement WM for a continuum of possible spatial locations over several seconds and have been proposed for the organization of prefrontal cortical networks. I first present a novel method for the efficient prediction of the network-wide steady states from the underlying microscopic network properties. The method can be applied to predict and tune the "bump" shapes of continuous attractors implemented in networks of spiking neuron models connected by nonlinear synapses, which we demonstrate for saturating synapses involving NMDA receptors. In a second part, I investigate the computational role of short-term synaptic plasticity as a synaptic nonlinearity. Continuous attractor models are sensitive to the inevitable variability of biological neurons: variable neuronal firing and heterogeneous networks decrease the time that memories are accurately retained, eventually leading to a loss of memory functionality on behaviorally relevant time scales. In theory and simulations, I show that short-term plasticity can control the time scale of memory retention, with facilitation and depression playing antagonistic roles in controlling the drift and diffusion of locations in memory. Finally, we place quantitative constraints on the combination of synaptic and network parameters under which continuous attractors networks can implement reliable WM in cortical settings.

EPFL2017

Asynchronous Simulation of Neuronal Activity

Bruno Ricardo Da Cunha Magalhães

EPFL2019

Continuous attractor working memory and provenance of channel models

Alexander Kevin Seeholzer

EPFL2017