Champ aléatoire conditionnel | EPFL Graph Search

Les champs aléatoires conditionnels (conditional random fields ou CRFs) sont une classe de modèles statistiques utilisés en reconnaissance des formes et plus généralement en apprentissage statistique. Les CRFs permettent de prendre en compte l'interaction de variables « voisines ». Ils sont souvent utilisés pour des données séquentielles (langage naturel, séquences biologiques, vision par ordinateur). Les CRFs sont un exemple de réseau probabiliste non orienté. Il s'agit d'une version de type modèle discriminant des champs aléatoires de Markov généralement présentés comme des modèles discriminatifs, c'est-à-dire que l'on cherche à modéliser la probabilité conditionnelle p(\boldsymbol{Y} |\boldsymbol{X}), \boldsymbol{X} étant les observations et \boldsymbol{Y} les variables à estimer, au lieu de p(\boldsymbol{Y}, \boldsymbol{X}). Description Un CRF est défini par Lafferty, McCallum et Pereira comme suit. Soit G = (

Localization errors of the stochastic heat equation

David Jean-Michel Candil

In this thesis, we study the stochastic heat equation (SHE) on bounded domains and on the whole Euclidean space

\R^d.

We confirm the intuition that as the bounded domain increases to the whole space, both solutions become arbitrarily close to one another. Both vanishing Dirichlet and Neumann boundary conditions are considered.We first study the nonlinear SHE in any space dimension with multiplicative correlated noise and bounded initial data. We prove that the solutions to SHE on an increasing sequence of domains converge exponentially fast to the solution to SHE on

\R^d.

Uniform convergence on compact set is obtained for all

p

-moments. The conditions that need to be imposed on the noise are the same as those required to ensure existence of a random field solution. A Gronwall-type iteration argument is used together with uniform bounds on the solutions, which are surprisingly valid for the entire sequence of increasing domains.We then study SHE in space dimension

d\ge 2

with additive white noise and bounded initial data. Even though both solutions need to be considered as distributions, their difference is proved to be smooth. If fact, the order of smoothness depends only on the regularity of the boundary of the increasing sequence of domains. We prove that the Fourier transform, in the sense of distributions, of the solution to SHE on

\R^d

do not have any locally mean-square integrable representative. Therefore, convergence is studied in local versions of Sobolev spaces. Again, exponential rate is obtained.Finally, we study the Anderson model for SHE with correlated noise and initial data given by a measure. We obtain a special expression for the second moment of the difference of the solution on

\R^d

with that on a bounded domain. The contribution of the initial condition is made explicit. For example, exponentially fast convergence on compact sets is obtained for any initial condition with polynomial growth. More interestingly, from a given convergence rate, we can decide whether some initial data is admissible.

EPFL2022

Conditional Random Fields for Multi-Camera Object Detection

Horesh Beny Ben Shitrit, Xavier Boix Bosch, Pascal Fua, Gemma Roig Noguera

We formulate a model for multi-class object detection in a multi-camera environment. From our knowledge, this is the first time that this problem is addressed taken into account different object classes simultaneously. Given several images of the scene taken from different angles, our system estimates the ground plane location of the objects from the output of several object detectors applied at each viewpoint. We cast the problem as an energy minimization modeled with a Conditional Random Field (CRF). Instead of predicting the presence of an object at each image location independently, we simultaneously predict the labeling of the entire scene. Our CRF is able to take into account occlusions between objects and contextual constraints among them. We propose an effective iterative strategy that renders tractable the underlying optimization problem, and learn the parameters of the model with the max-margin paradigm. We evaluate the performance of our model on several challenging multi-camera pedestrian detection datasets namely PETS 2009 and EPFL terrace sequence. We also introduce a new dataset in which multiple classes of objects appear simultaneously in the scene. It is here where we show that our method effectively handles occlusions in the multi-class case.

2011

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, our understanding of these models, in particular of what happens during their training, remains very limited. This PhD started with the introduction of the Neural Tangent Kernel (NTK) to describe the evolution of the function represented by the network during training. In the infinite-width limit, i.e. when the number of neurons in the layers of the network grows to infinity, the NTK converges to a deterministic and time-independent limit, leading to a simple yet complete description of the dynamics of infinitely-wide DNNs. This allowed one to give the first general proof of convergence of DNNs to a global minimum, and yielded the first description of the limiting spectrum of the Hessian of the loss surface of DNNs throughout training.More importantly, the NTK plays a crucial role in describing the generalization abilities of DNNs, i.e. the performance of the trained network on unseen data. The NTK analysis uncovered a direct link between the function learned by infinitely wide DNNs and Kernel Ridge Regression predictors, whose generalization properties are studied in this thesis using tools of random matrix theory. Our analysis of KRR reveals the importance of the eigendecomposition of the NTK, which is affected by a number of architectural choices. In very deep networks, an ordered regime and a chaotic regime appear, determined by the choice of non-linearity and the balance between the weights and bias parameters; these two phases are characterized by different speeds of decay of the eigenvalues of the NTK, leading to a tradeoff between convergence speed and generalization. In practical contexts such as Generative Adversarial Networks or Topology Optimization, the network architecture can be chosen to guarantee certain properties of the NTK and its spectrum.These results give an almost complete description DNNs in this infinite-width limit. It is then natural to wonder how it extends to finite-width networks used in practice. In the so-called NTK regime, the discrepancy between finite- and infinite-widths DNNs is mainly a result of the variance w.r.t. to the sampling of the parameters, as shown empirically and mathematically relying on the similarity between DNNs and random feature models.In contrast to the NTK regime, where the NTK remains constant during training, there exist so-called active regimes, where the evolution of the NTK is significant, which appear in a number of settings. One such regime appears in Deep Linear Networks with a very small initialization, where the training dynamics approach a sequence of saddle-points, representing linear maps of increasing rank, leading to a low-rank bias which is absent in the NTK regime.

EPFL2022

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

EPFL2022

Localization errors of the stochastic heat equation

David Jean-Michel Candil

In this thesis, we study the stochastic heat equation (SHE) on bounded domains and on the whole Euclidean space

\R^d.

\R^d.

Uniform convergence on compact set is obtained for all

p

d\ge 2

\R^d

\R^d

EPFL2022