PCM for Neuromorphic Applications: Impact of Device Characteristics on Neural Network Performance

Irem Boybat Kara, Carmelo Di Nolfo, Yusuf Leblebici, Severin Sidler
2015
Article de conférence

Résumé

The impact of Phase Change Memory (PCM) as well as other Non-Volatile Memory (NVM) device characteristics on the quantitative classification performance of artificial neural networks is studied. Our results show that any NVM-based neural network — not just those based on PCM — can be expected to be highly resilient to random effects (device variability, yield, and stochasticity), but will be highly sensitive to “gradient” effects that act to steer all synaptic weights. Asymmetry, such as that found with PCM, can be mitigated by an occasional RESET strategy, which can be both infrequent and inaccurate. Algorithms that can finesse some of the imperfections of NVM devices are proposed.

Official source

https://infoscience.epfl.ch/record/211159?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Publications associées (5)

, , ,

We assess the impact of the conductance response of Non-Volatile Memory (NVM) devices employed as the synaptic weight element for on-chip acceleration of the training of large-scale artificial neural networks (ANN). We briefly review our previous work towards achieving competitive performance (classification accuracies) for such ANN with both Phase-Change Memory (PCM) [1], [2] and non-filamentary ReRAM based on PrCaMnO (PCMO) [3], and towards assessing the potential advantages for ML training over GPU–based hardware in terms of speed (up to 25x faster) and power (from 120–2850x lower power) [4]. We then discuss the “jump-table” concept, previously introduced to model real-world NVM such as PCM [1] or PCMO, to describe the full cumulative distribution function (CDF) of conductance-change at each device conductance value, for both potentiation (SET) and depression (RESET). Using several types of artificially–constructed jump-tables, we assess the relative importance of deviations from an ideal NVM with perfectly linear conductance response.

Ieee2016

Chargement

Neuromorphic Computing Using Non-Volatile Memory

Irem Boybat Kara, Alessandro Fumarola, Seyoung Kim, Yusuf Leblebici, Abu Sebastian, Severin Sidler

Dense crossbar arrays of non-volatile memory (NVM) devices represent one possible path for implementing massively-parallel and highly energy-efficient neuromorphic computing systems. We first review recent advances in the application of NVM devices to three computing paradigms: spiking neural networks (SNNs), deep neural networks (DNNs), and ‘Memcomputing’. In SNNs, NVM synaptic connections are updated by a local learning rule such as spike-timing-dependent-plasticity, a computational approach directly inspired by biology. For DNNs, NVM arrays can represent matrices of synaptic weights, implementing the matrix–vector multiplication needed for algorithms such as backpropagation in an analog yetmassively-parallel fashion. This approach could provide significant improvements in power and speed compared to GPU-based DNN training, for applications of commercial significance. We then survey recent research in which different types of NVM devices – including phase change memory, conductive-bridging RAM, filamentary and nonfilamentary RRAM, and other NVMs – have been proposed, either as a synapse or as a neuron, for use within a neuromorphic computing application. The relevant virtues and limitations of these devices are assessed, in terms of properties such as conductance dynamic range, (non)linearity and (a)symmetry of conductance response, retention, endurance, required switching power, and device variability.

Taylor & Francis Ltd2016

Chargement

Non-volatile memory as hardware synapse in neuromorphic computing: A first look at reliability issues

Irem Boybat Kara, Carmelo Di Nolfo

A large-scale artificial neural network, a three-layer perceptron, is implemented using two phase-change memory (PCM) devices to encode the weight of each of 164,885 synapses. The PCM conductances are programmed using a crossbar-compatible pulse scheme, and the network is trained to recognize a 5000-example subset of the MNIST handwritten digit database, achieving 82.2% accuracy during training and 82.9% generalization accuracy on unseen test examples. A simulation of the network performance is developed that incorporates a statistical model of the PCM response, allowing quantitative estimation of the tolerance of the network to device variation, defects, and conductance response.

IEEE2015

Chargement

Afficher plus