**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Publication# An efficient initialization mechanism of neurons for Winner Takes All Neural Network implemented in the CMOS technology

Abstract

The paper presents a new initialization mechanism based on a Convex Combination Method (CCM) for Kohonen self-organizing Neural Networks (NNs) realized in the CMOS technology. A proper selection of initial values of the neuron weights exhibits a strong impact on the quality of the overall learning process. Unfortunately, in case of real input data, e.g. biomedical data, proper initialization is not easy to perform, as an exact data distribution is usually unknown. Bad initialization causes that even 70%-80% of neurons remain inactive, which increases the quantization error and thus limits the classification abilities of the NN. The proposed initialization algorithm has a couple of important advantages. Firstly, it does not require a knowledge of data distribution in the input data space. Secondly, there is no necessity for an initial polarization of the neuron weights before starting the learning process. This feature is very convenient in case of transistor level realizations. In this case the programming lines, which in other approaches occupy a large chip area, are not required. We proposed a modification of the original CCM algorithm. A new parameter which in the proposed analog CMOS realization is represented by an external current, allows to fit the behavior of the mechanism to NNs containing different numbers of neurons. The investigations show that the modified CCM operates properly for the NN containing even 250 neurons. A single CCM block realized in the CMOS 180 nm technology occupies an area of 300 mu m(2) and dissipates an average power of 20 mu W and at data rate of up to 20 MHz. (C) 2015 Elsevier Inc. All rights reserved.

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related MOOCs (32)

Related publications (43)

Related concepts (33)

Neuronal Dynamics - Computational Neuroscience of Single Neurons

The activity of neurons in the brain and the code used by these neurons is described by mathematical neuron models at different levels of detail.

Neuronal Dynamics - Computational Neuroscience of Single Neurons

The activity of neurons in the brain and the code used by these neurons is described by mathematical neuron models at different levels of detail.

Neuronal Dynamics 2- Computational Neuroscience: Neuronal Dynamics of Cognition

This course explains the mathematical and computational models that are used in the field of theoretical neuroscience to analyze the collective dynamics of thousands of interacting neurons.

Normal distribution

In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution (and also its median and mode), while the parameter is its standard deviation. The variance of the distribution is . A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Martin Louis Lucien Rémy Barry

Humans and animals constantly adapt to their environment over the course of their life. This thesis seeks to integrate various timescales of adaptation, ranging from the adaptation of synaptic connections between spiking neurons (milliseconds), rapid behav ...

Shuqing Teresa Yeo, Amir Roshan Zamir, Oguzhan Fatih Kar, Zahra Sodagar

We propose a method for adapting neural networks to distribution shifts at test-time. In contrast to training-time robustness mechanisms that attempt to anticipate and counter the shift, we create a closed-loop system and make use of test-time feedback sig ...

Michaël Unser, Shayan Aziznejad, Joaquim Gonçalves Garcia Barreto Campos

We develop a novel 2D functional learning framework that employs a sparsity-promoting regularization based on second-order derivatives. Motivated by the nature of the regularizer, we restrict the search space to the span of piecewise-linear box splines shi ...