Simulation Neurotechnologies for Advancing Brain Research: Parallelizing Large Networks in NEURON

Large multiscale neuronal network simulations are of increasing value as more big data are gathered about brain wiring and organization under the auspices of a current major research initiative, such as Brain Research through Advancing Innovative Neurotechnologies. The development of these models requires new simulation technologies. We describe here the current use of the NEURON simulator with message passing interface (MPI) for simulation in the domain of moderately large networks on commonly available high-performance computers (HPCs). We discuss the basic layout of such simulations, including the methods of simulation setup, the run-time spike-passing paradigm, and postsimulation data storage and data management approaches. Using the Neuroscience Gateway, a portal for computational neuroscience that provides access to large HPCs, we benchmark simulations of neuronal networks of different sizes (500-100,000 cells), and using different numbers of nodes (1-256). We compare three types of networks, composed of either Izhikevich integrate-and-fire neurons (I&F), single-compartment Hodgkin-Huxley (HH) cells, or a hybrid network with half of each. Results show simulation run time increased approximately linearly with network size and decreased almost linearly with the number of nodes. Networks with I&F neurons were faster than HH networks, although differences were small since all tested cells were point neurons with a single compartment.

Biologically plausible unsupervised learning in shallow and deep neural networks

Bernd Albert Illing

The way our brain learns to disentangle complex signals into unambiguous concepts is fascinating but remains largely unknown. There is evidence, however, that hierarchical neural representations play a key role in the cortex. This thesis investigates biologically plausible models of unsupervised learning of hierarchical representations as found in the brain and modern computer vision models. We use computational modeling to address three main questions at the intersection of artificial intelligence (AI) and computational neuroscience.The first question is: What are useful neural representations and when are deep hierarchical representations needed? We approach this point with a systematic study of biologically plausible unsupervised feature learning in a shallow 2-layer networks on digit (MNIST) and object (CIFAR10) classification. Surprisingly, random features support high performance, especially for large hidden layers. When combined with localized receptive fields, random feature networks approach the performance of supervised backpropagation on MNIST, but not on CIFAR10. We suggest that future models of biologically plausible learning should outperform such random feature benchmarks on MNIST, or that such models should be evaluated in different ways.The second question is: How can hierarchical representations be learned with mechanisms supported by neuroscientific evidence? We cover this question by proposing a unifying Hebbian model, inspired by common models of V1 simple and complex cells based on unsupervised sparse coding and temporal invariance learning. In shallow 2-layer networks, our model reproduces learning of simple and complex cell receptive fields, as found in V1. In deeper networks, we stack multiple layers of Hebbian learning but find that it does not yield hierarchical representations of increasing usefulness. From this, we hypothesise that standard Hebbian rules are too constrained to build increasingly useful representations, as observed in higher areas of the visual cortex or deep artificial neural networks.The third question is: Can AI inspire learning models that build deep representations and are still biologically plausible? We address this question by proposing a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. The proposed rule is Hebbian, i.e. only depends on pre- and post-synaptic neuronal activity, but includes additional local factors, namely predictive dendritic input and widely broadcasted modulation factors. Algorithmically, this rule applies self-supervised contrastive predictive learning to a causal, biological setting using saccades. We find that networks trained with this generalised Hebbian rule build deep hierarchical representations of images, speech and video.We see our modeling as a potential starting point for both, new hypotheses, that can be tested experimentally, and novel AI models that could benefit from added biological realism.

EPFL2021

Simple Neuron Models and Adaptation

Skander Mensi

Predicting activity of single neuron is an important part of the computational neuroscience and a great challenge. Several mathematical models exist, from the simple (one compartment and few parameters, like the SRM or the IF-type models), to the more complex (biophysical model like the Hodgkin-Huxley model). All these models have their own advantages and limitations, but no one is able to reproduce the exact behavior of real neurons. Multiple projects try to simulate complex neural networks, or even the whole brain (i.e. the blue brain project or other big network simulation). To achieve this goal it is very important that simple neuron models simulate, for a low computational cost, the precise activities of all neuron classes. It is well-known that neurons exhibit a lot of different activity patterns (from an electro-physiological point of view). Here we have focused on pyramidal neurons from the layer 5 of the neocortex. They are classified as regular spiking-cells. This cell type shows adaptation and like other neurons, refractoriness. Adaptation and refractoriness are very common neuronal activity in the brain, and so it is important to have a simple model which can reproduce this kind of activity. This project deals with two classical simple neuron models: the adaptive exponential integrate-and-fire model (AdEx) and the spike response model (SRM). We deter- mined the parameters of these two models using data generated with a detailed model, the Destexhe's model which is a HH-like model for cortical pyramidal cells, stimulated with different current injection scenarios. In a second time the model parameters have been set using data from in-vitro recordings of 4 layer 5 pyramidal rat neurons, stimulated with a sinusoid in vivo-like protocol, injected somatically in current-clamp configuration. Then we show that this type of model can capture adaptation and can reproduce the activity of neuron with a high reliability

2008

PyNEST: a convenient interface to the NEST simulator

Marc-Oliver Gewaltig, Eilif Benjamin Muller

The neural simulation tool NEST (http://www.nest-initiative.org) is a simulator for heterogeneous networks of point neurons or neurons with a small number of compartments. It aims at simulations of large neural systems with more than 10^4 neurons and 10^7 to 10^9 synapses. NEST is implemented in C++ and can be used on a large range of architectures from single-core laptops over multi-core desktop computers to super-computers with thousands of processor cores. Python (http://www.python.org) is a modern programming language that has recently received considerable attention in Computational Neuroscience. Python is easy to learn and has many extension modules for scientific computing (e.g. http://www.scipy.org). In this contribution we describe PyNEST, the new user interface to NEST. PyNEST combines NEST’s efficient simulation kernel with the simplicity and flexibility of Python. Compared to NEST’s native simulation language SLI, PyNEST makes it easier to set up simulations, generate stimuli, and analyze simulation results. We describe how PyNEST connects NEST and Python and how it is implemented. With a number of examples, we illustrate how it is used.