Carte autoadaptative | EPFL Graph Search

Les cartes autoadaptatives, cartes auto-organisatrices ou cartes topologiques forment une classe de réseau de neurones artificiels fondée sur des méthodes d'apprentissage non supervisées. Elles sont souvent désignées par le terme anglais self organizing maps (SOM), ou encore cartes de Kohonen du nom du statisticien ayant développé le concept en 1984. La littérature utilise aussi les dénominations : « réseau de Kohonen », « réseau autoadaptatif » ou « réseau autoorganisé ». Elles sont utilisées pour cartographier un espace réel, c'est-à-dire pour étudier la répartition de données dans un espace à grande dimension. En pratique, cette cartographie peut servir à réaliser des tâches de discrétisation, quantification vectorielle ou classification. Idée de base Ces structures intelligentes de représentation de données sont inspirées, comme beaucoup d’autres créations de l’intelligence artificielle, par la biologie. Il s'agit de reproduire le principe neuronal du cerveau des v

Clustering of winter precipitations in Valais

Clémence Marie Anne Le Cornec

This master thesis provides an extended study of winter precipitations in Valais (Switzerland). It was rst tried to relate local conditions in Valais, as measured at gauges, to the Hess-Brezowski classication for Europe. It was not possible to clearly relate this classication to local precipitation events in Winter in this region. Self-organizing maps were used to try to distinguish between sub-regions only based on the precipitation regime in Valais. The obtained results present coherence with the existing literature. ERA-Interim (global atmospheric reanalysis) data were then used to develop a new classication to link European meteorological conditions to local events. Three unsupervised clustering techniques were used for this purpose: k-means, k-medoids, hierachical clustering. The optimal number of cluster was determined using a com- bination four commonly used internal clusters indices (Dunn index, Davies-Bouldin index, silhouette index and Calinski-Harabasz index). Two clusters appeared to be optimal compared to the various parameters of the problem. The outputs of the three classication techniques were compared in term of robustness and results. A physical interpretation was then made. It was found that the two classes were associated respectively with small or large precipitation events. The associated synoptic conditions at the European scale reveal that the small precipitation class is associated with a North-East ux over Switzerland whereas the large precipitation class is linked to a South-West ux linked to a stationary front over Switzerland.

2017

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same functional unit, the neuron, and develop through the same learning mechanism, synaptic plasticity. It motivates the conjecture of a unifying theory to explain cortical representational learning across sensory modalities. In this thesis we present theories and computational models of learning and optimization in neural networks, postulating functional properties of synaptic plasticity that support the apparent universal learning capacity of cortical networks. In the past decades, a variety of theories and models have been proposed to describe receptive field formation in sensory areas. They include normative models such as sparse coding, and bottom-up models such as spike-timing dependent plasticity. We bring together candidate explanations by demonstrating that in fact a single principle is sufficient to explain receptive field development. First, we show that many representative models of sensory development are in fact implementing variations of a common principle: nonlinear Hebbian learning. Second, we reveal that nonlinear Hebbian learning is sufficient for receptive field formation through sensory inputs. A surprising result is that our findings are independent of specific details, and allow for robust predictions of the learned receptive fields. Thus nonlinear Hebbian learning and natural statistics can account for many aspects of receptive field formation across models and sensory modalities. The Hebbian learning theory substantiates that synaptic plasticity can be interpreted as an optimization procedure, implementing stochastic gradient descent. In stochastic gradient descent inputs arrive sequentially, as in sensory streams. However, individual data samples have very little information about the correct learning signal, and it becomes a fundamental problem to know how many samples are required for reliable synaptic changes. Through estimation theory, we develop a novel adaptive learning rate model, that adapts the magnitude of synaptic changes based on the statistics of the learning signal, enabling an optimal use of data samples. Our model has a simple implementation and demonstrates improved learning speed, making this a promising candidate for large artificial neural network applications. The model also makes predictions on how cortical plasticity may modulate synaptic plasticity for optimal learning. The optimal sampling size for reliable learning allows us to estimate optimal learning times for a given model. We apply this theory to derive analytical bounds on times for the optimization of synaptic connections. First, we show this optimization problem to have exponentially many saddle-nodes, which lead to small gradients and slow learning. Second, we show that the number of input synapses to a neuron modulates the magnitude of the initial gradient, determining the duration of learning. Our final result reveals that the learning duration increases supra-linearly with the number of synapses, suggesting an effective limit on synaptic connections and receptive field sizes in developing neural networks.

EPFL2016

LSSANet: A Long Short Slice-Aware Network for Pulmonary Nodule Detection

Bo Du, Rui Xu, Jiancheng Yang

Convolutional neural networks (CNNs) have been demonstrated to be highly effective in the field of pulmonary nodule detection. However, existing CNN based pulmonary nodule detection methods lack the ability to capture long-range dependencies, which is vital for global information extraction. In computer vision tasks, non-local operations have been widely utilized, but the computational cost could be very high for 3D computed tomography (CT) images. To address this issue, we propose a long short slice-aware network (LSSANet) for the detection of pulmonary nodules. In particular, we develop a new non-local mechanism termed long short slice grouping (LSSG), which splits the compact non-local embeddings into a short-distance slice grouped one and a long-distance slice grouped counterpart. This not only reduces the computational burden, but also keeps long-range dependencies among any elements across slices and in the whole feature map. The proposed LSSG is easy-to-use and can be plugged into many pulmonary nodule detection networks. To verify the performance of LSSANet, we compare with several recently proposed and competitive detection approaches based on 2D/3D CNN. Promising evaluation results on the large-scale PN9 dataset demonstrate the effectiveness of our method. Code is at https:// github.com/Ruixxxx/LSSANet.

SPRINGER INTERNATIONAL PUBLISHING AG2022

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

EPFL2016