Probabilité a priori | EPFL Graph Search

Dans le théorème de Bayes, la probabilité a priori (ou prior) désigne une probabilité se fondant sur des données ou connaissances antérieures à une observation. Elle s'oppose à la probabilité a posteriori (ou posterior) correspondante qui s'appuie sur les connaissances postérieures à cette observation. Formalisation Théorème de Bayes Le théorème de Bayes s'énonce de la manière suivante : :\mathbb{P}(A|B) = \frac{\mathbb{P}(B | A)\cdot \mathbb{P}(A)}{\mathbb{P}(B)}, si \mathbb{P}(B)\neq0. \mathbb{P}(A) désigne ici la probabilité a priori de A , tandis que \mathbb{P}(A|B) désigne la probabilité a posteriori, c'est-à-dire la probabilité conditionnelle de A sachant B . Lois Soit θ un paramètre ou vecteur de paramètres inconnu considéré aléatoire :

la loi de la variable aléatoire \theta avant observation est a

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks.

Konstantinos Pitas

Deep neural networks have been empirically successful in a variety of tasks, however their theoretical understanding is still poor. In particular, modern deep neural networks have many more parameters than training data. Thus, in principle they should overfit the training samples and exhibit poor generalization to the complete data distribution. Counter intuitively however, they manage to achieve both high training accuracy and high testing accuracy. One can prove generalization using a validation set, however this can be difficult when training samples are limited and at the same time we do not obtain any information about why deep neural networks generalize well. Another approach is to estimate the complexity of the deep neural network. The hypothesis is that if a network with high training accuracy has high complexity it will have memorized the data, while if it has low complexity it will have learned generalizable patterns. In the first part of this thesis we explore Spectral Complexity, a measure of complexity that depends on combinations of norms of the weight matrices of the deep neural network. For a dataset that is difficult to classify, with no underlying model and/or no recurring pattern, for example one where the labels have been chosen randomly, spectral complexity has a large value, reflecting that the network needs to memorize the labels, and will not generalize well. Putting back the real labels, the spectral complexity becomes lower reflecting that some structure is present and the network has learned patterns that might generalize to unseen data. Spectral complexity results in vacuous estimates of the generalization error (the difference between the training and testing accuracy), and we show that it can lead to counterintuitive results when comparing the generalization error of different architectures. In the second part of the thesis we explore non-vacuous estimates of the generalization error. In Chapter 2 we analyze the case of PAC-Bayes where a posterior distribution over the weights of a deep neural network is learned using stochastic variational inference, and the generalization error is the KL divergence between this posterior and a prior distribution. We find that a common approximation where the posterior is constrained to be Gaussian with diagonal covariance, known as the mean-field approximation, limits significantly any gains in bound tightness. We find that, if we choose the prior mean to be the random network initialization, the generalization error estimate tightens significantly. In Chapter 3 we explore an existing approach to learning the prior mean, in PAC-Bayes, from the training set. Specifically, we explore differential privacy, which ensures that the training samples contribute only a limited amount of information to the prior, making it distribution and not training set dependent. In this way the prior should generalize well to unseen data (as it hasn't memorized individual samples) and at the same time any posterior distribution that is close to it in terms of the KL divergence will also exhibit good generalization.

EPFL2020

Limit Behaviour of the Voter Model, Exclusion Process and Regenerative Chains

Samuel Schöpfer

Though the following topics seem unlinked, most of the tools used in this thesis are related to random walks and renewal theory. After introducing the voter model, we consider the parabolic Anderson model with the voter model as catalyst. In GÄRTNER, DEN HOLLANDER and MAILLARD [44], the behaviour of the annealed Lyapunov exponents, i.e., the exponential growth rates of the successive moments of the reactant with respect to the catalyst, was investigated. It was shown that these exponents exhibit an interesting dependence on the dimension and on the diffusion constant. In Chapter 3 we address some questions left open in this paper by considering specifically when the Lyapunov exponents are the a priori maximal value. Then, we use exclusion process techniques to show that the evolution of a perturbed threshold voter model is recurrent in the critical case. The key to our approach is to develop the ideas of BRAMSON and MOUNTFORD [9] : we exhibit a Lyapunov-Foster function for the discrete time version of the process. We also make a widespread use of coupling arguments. Finally, using the regenerative scheme of COMETS, FERNÁNDEZ and FERRARI [19], we establish a functional central limit theorem for discrete time stochastic processes with summable memory decay. Furthermore, under stronger assumptions on the memory decay, we identify the limiting variance in terms of the process only. As applications, we define classes of binary autoregressive processes and power-law Ising chains for which the limit theorem is fulfilled.

EPFL2012

Morphology-driven automatic segmentation of MR images of the neonatal brain

Laura Ioana Gui, Michel Kocher, François Lazeyras, Radoslaw Lisowski

The segmentation of MR images of the neonatal brain is an essential step in the study and evaluation of infant brain development. State-of-the-art methods for adult brain MRI segmentation are not applicable to the neonatal brain, due to large differences in structure and tissue properties between newborn and adult brains. Existing newborn brain MRI segmentation methods either rely on manual interaction or require the use of atlases or templates, which unavoidably introduces a bias of the results towards the population that was used to derive the atlases. We propose a different approach for the segmentation of neonatal brain MRI, based on the infusion of high-level brain morphology knowledge, regarding relative tissue location, connectivity and structure. Our method does not require manual interaction, or the use of an atlas, and the generality of its priors makes it applicable to different neonatal populations, while avoiding atlas-related bias. The proposed algorithm segments the brain both globally (intracranial cavity, cerebellum, brainstem and the two hemispheres) and at tissue level (cortical and subcortical gray matter, myelinated and unmyelinated white matter, and cerebrospinal fluid). We validate our algorithm through visual inspection by medical experts, as well as by quantitative comparisons that demonstrate good agreement with expert manual segmentations. The algorithm's robustness is verified by testing on variable quality images acquired on different machines, and on subjects with variable anatomy (enlarged ventricles, preterm- vs. term-born). (C) 2012 Elsevier B.V. All rights reserved.

Elsevier Science Bv2012

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks.

Konstantinos Pitas

EPFL2020