Neuromorphic engineering | EPFL Graph Search

Neuromorphic computing is an approach to computing that is inspired by the structure and function of the human brain. A neuromorphic computer/chip is any device that uses physical artificial neurons to do computations. In recent times, the term neuromorphic has been used to describe analog, digital, mixed-mode analog/digital VLSI, and software systems that implement models of neural systems (for perception, motor control, or multisensory integration). The implementation of neuromorphic computing on the hardware level can be realized by oxide-based memristors, spintronic memories, threshold switches, transistors, among others. Training software-based neuromorphic systems of spiking neural networks can be achieved using error backpropagation, e.g., using Python based frameworks such as snnTorch, or using canonical learning rules from the biological learning literature, e.g., using BindsNet. A key aspect of neuromorphic engineering is understanding how the morphology of

All-memristive neuromorphic computing with level-tuned neurons

Evangelos Eleftheriou, Angeliki Pantazi, Stanislaw Andrzej Wozniak

In the new era of cognitive computing, systems will be able to learn and interact with the environment in ways that will drastically enhance the capabilities of current processors, especially in extracting knowledge from vast amount of data obtained from many sources. Brain-inspired neuromorphic computing systems increasingly attract research interest as an alternative to the classical von Neumann processor architecture, mainly because of the coexistence of memory and processing units. In these systems, the basic components are neurons interconnected by synapses. The neurons, based on their nonlinear dynamics, generate spikes that provide the main communication mechanism. The computational tasks are distributed across the neural network, where synapses implement both the memory and the computational units, by means of learning mechanisms such as spike-timing-dependent plasticity. In this work, we present an all-memristive neuromorphic architecture comprising neurons and synapses realized by using the physical properties and state dynamics of phase-change memristors. The architecture employs a novel concept of interconnecting the neurons in the same layer, resulting in level-tuned neuronal characteristics that preferentially process input information. We demonstrate the proposed architecture in the tasks of unsupervised learning and detection of multiple temporal correlations in parallel input streams. The efficiency of the neuromorphic architecture along with the homogenous neuro-synaptic dynamics implemented with nanoscale phase-change memristors represent a significant step towards the development of ultrahigh-density neuromorphic co-processors.

Iop Publishing Ltd2016

Unsupervised Learning of Phase-Change-Based Neuromorphic Systems

Stanislaw Andrzej Wozniak

Neuromorphic systems provide brain-inspired methods of computing. In a neuromorphic architecture, inputs are processed by a network of neurons receiving operands through synaptic interconnections, tuned in the process of learning. Neurons act simultaneously as asynchronous computational and memory units, which leads to a high degree of parallelism. Furthermore, owing to developments in novel materials, memristive devices were proposed for area- and energy-efficient mixed digital-analog implementation of neurons and synapses. In this dissertation, we propose neuromorphic architectures based on phase-change memristors combined with biologically-inspired synaptic learning rules, and we experimentally demonstrate their pattern- and feature-learning capabilities. Firstly, by exploiting the physical properties of phase-change devices, we propose neuromorphic building blocks comprising phase-change-based neurons and synapses operating according to an unsupervised local learning rule. At the same time, we introduce multiple enhancements for pattern learning: an integration threshold for the phase-change soma to ensure noise-robust operation; selective synaptic depression mechanism to limit negative impact of asymmetric conductance response of phase-change synapses during the learning; WTA (Winner-Take-All) mechanism with level-tuned neurons that decreases power consumption in comparison to the classic lateral inhibition WTA; and learning WTA that enhances the quality of pattern visualization. Experimental results demonstrate the capabilities of the proposed architectures. In particular, a neuron with phase-change synapses was shown to learn and re-learn patterns of correlated activity. Furthermore, an all-phase-change neuron with a record number of 1M synapses successfully detected and visualized weakly-correlated patterns. Lastly, a network of all-phase-change neurons operating with level-tuned neurons accurately learned multiple patterns. Secondly, to scale-up the proposed architectures, we identify the need to improve the knowledge representation to learn features rather than patterns. We determine the key role of the feedback links for controlling the learning process, and combine intraneuronal with interneuronal feedback. Intraneuronal feedback determines what each neuron learns, whereas interneuronal feedback determines how information is distributed between the neurons. We propose two feature-learning architectures: an architecture with interneuronal feedback to the learning rule, and an architecture inspired by the biological observation of synaptic competition for learning-related proteins. Furthermore, we introduce a model of synaptic competition that guides the learning as well as detects novelty in the input, which is then used to dynamically adjust the size of the network. In a series of benchmarks for different feature types, synaptic competition outperformed other common methods, simultaneously adjusting the network to the optimal size. Finally, it was the only method that succeeded for a challenging dataset that violates the common machine learning assumption on the independent and identically-distributed input presentation. To conclude, we proposed phase-change-based neuromorphic architectures and realized them in a large-scale prototype platform. The experimental results demonstrate pattern- and feature-learning capabilities and constitute an important step towards designing unsupervised online learning neuromorphic systems.

EPFL2017

Networks of Coupled VO2 Oscillators for Neuromorphic Computing

Elisabetta Corti

Neuromorphic computing is a wide research field aimed to the realization of brain-inspired hardware, apt to tackle computation of unstructured data more efficiently than currently done with standard computational units. Oscillatory neural networks are known for their associative memory capability, which enables to retrieve the information stored in the system from noisy or incomplete data. The development of phase-transition materials such as vanadiumdioxide (VO2) allows to design compact relaxation oscillator units which can be coupled in frequency and phase to realize an oscillatory neural network in hardware. In this thesis, we investigate the oscillatory neural network technology from the realization of the basic oscillator components with VO2 to the exploitation of the coupled oscillators as analog filters in convolutional neural networks applications. VO2 phase-transition devices are realized in a CMOS compatible process in two geometries, a planar and a crossbar configuration. The impact of the polycrystallinity of the VO2 film on the insulator-to-metal transition of the device is analyzed; through the contacting of a single grain we demonstrate the realization of a VO2 device with a single, sharp phase transition. The VO2 devices are connected in circuits to build networks of coupled oscillators. Through coupling with resistive and capacitive elements, experimental demonstrations of a 4-VO2 coupled oscillator network is shown. The network encodes the input and output information in the relative phase of the oscillators. The associative memory capability of the system is used to extract features from hand-written digits. By expanding the network to a 3×3 coupled oscillator system, we demonstrate in simulations how an oscillatory neural network can replace up to five digital filters in a convolutional neural network, retaining the same image processing capabilities.

EPFL2021

Unsupervised Learning of Phase-Change-Based Neuromorphic Systems

Stanislaw Andrzej Wozniak

EPFL2017