Méthode des poids multiplicatifs

Résumé

La méthode des poids multiplicatifs ou multiplicative weight update method en anglais, est une méthode algorithmique. C'est un méta-algorithme probabiliste qui apparaît dans de nombreux domaines sous diverses formes et divers noms, par exemple l'algorithme en théorie des jeux et l'algorithme Adaboost en apprentissage automatique. Elle est utilisée dans de nombreux domaines de l'informatique théorique, comme la géométrie algorithmique, les algorithmes en ligne, la dérandomisation et l'optimisation linéaire. Principe Comme l'algorithme est générique, sa description et son contexte d'utilisation sont vagues et doivent être précisés pour chaque application. Contexte Le contexte peut être décrit de la manière suivante. Il y a une série de choix à faire, les uns après les autres. Après chaque décision, le coût de chaque option est donné, et il est donc possible de savoir à quel point le choix fait est bon ou non. Pour prendre ces décisions, il y a n experts, donnant

Source officielle

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées

Chargement

Personnes associées

Chargement

Unités associées

Chargement

Concepts associés

Chargement

Cours associés

Chargement

Séances de cours associées

Chargement

Weight Initialization for High Order and Multilayer Perceptrons

Proper weight initialization is one of the most important prerequisites for fast convergence of feed-forward neural networks like high order and multilayer perceptrons. In order to determine the optimal value of the initial weight variance (or range), which is a important parameter of random weight initialization methods for high order perceptrons, a wide range of experiments (more than

200,000

simulations) was performed, using seven different data sets, three weight distributions, three activation functions, and several network orders. The results of these experiments are compared to weight initialization techniques for multilayer perceptrons, which leads to the proposal of a suitable weight initialization method for high order perceptrons. Experiments over a large range of initial weight variances are performed (more than

20,000

simulations) for multilayer perceptrons and compared to weight initialization methods proposed by other authors. The results of this comparison are justified by sufficiently small confidence intervals.

1994

Chargement

Local plasticity rules can learn deep representations using self-supervised contrastive predictions

Wulfram Gerstner, Bernd Albert Illing, Jean Ventura

Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. Learning minimizes a simple layer-specific loss function and does not need to back-propagate error signals within or between layers. Instead, weight updates follow a local, Hebbian, learning rule that only depends on pre-and post-synaptic neuronal activity, predictive dendritic input and widely broadcasted modulation factors which are identical for large groups of neurons. The learning rule applies contrastive predictive learning to a causal, biological setting using saccades (ie rapid shifts in gaze direction). We find that networks trained with this self-supervised and local rule build deep hierarchical representations of images, speech and video.

2021

Chargement

Proper initialization is one of the most important prerequisites for fast convergence of feed-forward neural networks like high order and multilayer perceptrons. This publication aims at determining the optimal variance (or range) for the initial weights and biases, which is the principal parameter of random initialization methods for both types of neural networks. An overview of random weight initialization methods for multilayer perceptrons is presented. These methods are extensively tested using eight real- world benchmark data sets and a broad range of initial weight variances by means of more than

30,000

simulations, in the aim to find the best weight initialization method for multilayer perceptrons. For high order networks, a large number of experiments (more than

200,000

simulations) was performed, using three weight distributions, three activation functions, several network orders, and the same eight data sets. The results of these experiments are compared to weight initialization techniques for multilayer perceptrons, which leads to the proposal of a suitable initialization method for high order perceptrons. The conclusions on the initialization methods for both types of networks are justified by sufficiently small confidence intervals of the mean convergence times.

IEEE1997

Chargement

Weight Initialization for High Order and Multilayer Perceptrons

200,000

20,000

1994

30,000

simulations, in the aim to find the best weight initialization method for multilayer perceptrons. For high order networks, a large number of experiments (more than

200,000

IEEE1997