Unsupervised Learning of Phase-Change-Based Neuromorphic Systems

Neuromorphic systems provide brain-inspired methods of computing. In a neuromorphic architecture, inputs are processed by a network of neurons receiving operands through synaptic interconnections, tuned in the process of learning. Neurons act simultaneously as asynchronous computational and memory units, which leads to a high degree of parallelism. Furthermore, owing to developments in novel materials, memristive devices were proposed for area- and energy-efficient mixed digital-analog implementation of neurons and synapses. In this dissertation, we propose neuromorphic architectures based on phase-change memristors combined with biologically-inspired synaptic learning rules, and we experimentally demonstrate their pattern- and feature-learning capabilities. Firstly, by exploiting the physical properties of phase-change devices, we propose neuromorphic building blocks comprising phase-change-based neurons and synapses operating according to an unsupervised local learning rule. At the same time, we introduce multiple enhancements for pattern learning: an integration threshold for the phase-change soma to ensure noise-robust operation; selective synaptic depression mechanism to limit negative impact of asymmetric conductance response of phase-change synapses during the learning; WTA (Winner-Take-All) mechanism with level-tuned neurons that decreases power consumption in comparison to the classic lateral inhibition WTA; and learning WTA that enhances the quality of pattern visualization. Experimental results demonstrate the capabilities of the proposed architectures. In particular, a neuron with phase-change synapses was shown to learn and re-learn patterns of correlated activity. Furthermore, an all-phase-change neuron with a record number of 1M synapses successfully detected and visualized weakly-correlated patterns. Lastly, a network of all-phase-change neurons operating with level-tuned neurons accurately learned multiple patterns. Secondly, to scale-up the proposed architectures, we identify the need to improve the knowledge representation to learn features rather than patterns. We determine the key role of the feedback links for controlling the learning process, and combine intraneuronal with interneuronal feedback. Intraneuronal feedback determines what each neuron learns, whereas interneuronal feedback determines how information is distributed between the neurons. We propose two feature-learning architectures: an architecture with interneuronal feedback to the learning rule, and an architecture inspired by the biological observation of synaptic competition for learning-related proteins. Furthermore, we introduce a model of synaptic competition that guides the learning as well as detects novelty in the input, which is then used to dynamically adjust the size of the network. In a series of benchmarks for different feature types, synaptic competition outperformed other common methods, simultaneously adjusting the network to the optimal size. Finally, it was the only method that succeeded for a challenging dataset that violates the common machine learning assumption on the independent and identically-distributed input presentation. To conclude, we proposed phase-change-based neuromorphic architectures and realized them in a large-scale prototype platform. The experimental results demonstrate pattern- and feature-learning capabilities and constitute an important step towards designing unsupervised online learning neuromorphic systems.

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same functional unit, the neuron, and develop through the same learning mechanism, synaptic plasticity. It motivates the conjecture of a unifying theory to explain cortical representational learning across sensory modalities. In this thesis we present theories and computational models of learning and optimization in neural networks, postulating functional properties of synaptic plasticity that support the apparent universal learning capacity of cortical networks. In the past decades, a variety of theories and models have been proposed to describe receptive field formation in sensory areas. They include normative models such as sparse coding, and bottom-up models such as spike-timing dependent plasticity. We bring together candidate explanations by demonstrating that in fact a single principle is sufficient to explain receptive field development. First, we show that many representative models of sensory development are in fact implementing variations of a common principle: nonlinear Hebbian learning. Second, we reveal that nonlinear Hebbian learning is sufficient for receptive field formation through sensory inputs. A surprising result is that our findings are independent of specific details, and allow for robust predictions of the learned receptive fields. Thus nonlinear Hebbian learning and natural statistics can account for many aspects of receptive field formation across models and sensory modalities. The Hebbian learning theory substantiates that synaptic plasticity can be interpreted as an optimization procedure, implementing stochastic gradient descent. In stochastic gradient descent inputs arrive sequentially, as in sensory streams. However, individual data samples have very little information about the correct learning signal, and it becomes a fundamental problem to know how many samples are required for reliable synaptic changes. Through estimation theory, we develop a novel adaptive learning rate model, that adapts the magnitude of synaptic changes based on the statistics of the learning signal, enabling an optimal use of data samples. Our model has a simple implementation and demonstrates improved learning speed, making this a promising candidate for large artificial neural network applications. The model also makes predictions on how cortical plasticity may modulate synaptic plasticity for optimal learning. The optimal sampling size for reliable learning allows us to estimate optimal learning times for a given model. We apply this theory to derive analytical bounds on times for the optimization of synaptic connections. First, we show this optimization problem to have exponentially many saddle-nodes, which lead to small gradients and slow learning. Second, we show that the number of input synapses to a neuron modulates the magnitude of the initial gradient, determining the duration of learning. Our final result reveals that the learning duration increases supra-linearly with the number of synapses, suggesting an effective limit on synaptic connections and receptive field sizes in developing neural networks.

EPFL2016

Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity

Nicolas Frémaux, Wulfram Gerstner

Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term that captures the covariance of neuronal firing and reward and a second term that presents the influence of unsupervised learning. The unsupervised term, which is, in general, detrimental for reward-based learning, can be suppressed if the neuromodulatory signal encodes the difference between the reward and the expected reward—but only if the expected reward is calculated for each task and stimulus separately. If several tasks are to be learned simultaneously, the nervous system needs an internal critic that is able to predict the expected reward for arbitrary stimuli. We show that, with a critic, reward-modulated spike-timing-dependent plasticity is capable of learning motor trajectories with a temporal resolution of tens of milliseconds. The relation to temporal difference learning, the relevance of block-based learning paradigms, and the limitations of learning with a critic are discussed.

Society for Neuroscience2010

Network Activity and Plasticity

Vincent Delattre

The cortex must maintain balanced levels of neural activity to correctly integrate inputs and to provide contextually meaningful outputs. Neuronal excitation is counterbalanced by various forms of inhibition such as spike frequency adaptation, short- and long-term depression at excitatory synapses, and inhibitory cell coupling. When excitatory activity is low, homeostatic mechanisms enhance network excitability. When excitation is stronger than inhibition, network bursts of activity arise that also need to be dampened lest they run out of control. A disruption of the fine balance between excitatory and inhibitory systems is observed in an increasing numbers of brain disorders such as epilepsy, mental retardation and autism. This thesis aims to study aspects of the regulation of excitation in cortical networks at a molecular, cellular, network and disease level of investigation. We present in the first chapter an investigation of the impact of network bursts on standard spike-timing dependent plasticity (STDP) events in acute cortical brain slices. We discovered that these two kinds of events, which are the most well-established ways to induce synaptic plasticity in vitro, can block each other as well as cooperate to change the amplitude and direction of plasticity. Because the outcome of the interaction depends on the precise timing of network bursts relative to a STDP event in a single synaptic pathway, the observed network-timing dependent plasticity provides a novel mechanism for the modulation of local synaptic plasticity by the network level of activity, a form of global control of local synaptic plasticity. We hypothesized that network bursts consume extracellular resources needed to potentiate, restricting potentiation to active synapses at the burst onset while yielding depression of synapses activated at later times. As we demonstrate with simulation, this simple mechanism supports paradigm-shifting implications for models of learning, solving a persistent and fundamental problem since BCM (Bienenstock Cooper Munro, 1981) to incorporate synaptic learning into realistic neuronal networks while simultaneously establishing and maintaining excitatory-inhibitory balance. The proposed mechanism also links experimental evidence for interactions between STDP and network activity to observations of self-organized criticality in neural systems, a regime known to be optimal for information coding. In the second chapter, we investigated the effect of the loss of NLGN4 protein on synapse function and on network response to electrical stimulation. The neuroligin family of genes encodes for trans-synaptic proteins involved in synaptic differentiation and maturation. Former studies have reported that both NLGN1-KO and NLGN2-KO give rise to pathologic deregulations of the balance between excitation and inhibition in neural networks, and the presence of loss-of-function mutations in NLGN4 in a small number of autistic patients indicates a possible link between Neuroligin-4 gene and autism. Using extra-cellular stimulation and intra-cellular recordings in acute brain slices, we observed a decreased response of both excitatory and inhibitory response in NLGN4-KO animals, suggesting that NLGN4 is expressed at both excitatory and inhibitory synapses. Furthermore, we observed that the NLGN4-KO resulted in a significantly larger decrease of the excitatory response than of the inhibitory response, thus yielding a hypo-excitable network. The final chapter consists of the characterization of excitatory pathways in acute slices of the somato-sensory cortex. In the first study, we used chemical stimulation to promote the emergence of spontaneous recurrent activity and observed waves of activity originating from the cortical layer five. In the second study, we performed whole-cell patch-clamping of layer five pyramidal cells along with recording local field potentials (LFP). With the systematic layer-wise application of electrical stimulation at various frequencies coupled with LFP recordings, we obtained a frequency-dependent mapping of cortical inputs at all cortical layers at both the cellular and the network levels. Using this data, we developed a virtual framework for performing an analogous mapping in the virtual cortical column of the Blue Brain Project (BBP). This helped us understand the contribution of each cell population in the network response to stimulation. The framework also served as a validation test for the BBP cortical models.