**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Unit# Theory of statistical fields

Chair

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related people

Loading

Units doing similar research

Loading

Related research domains

Loading

Related publications

Loading

Related people (10)

Related publications (70)

Loading

Loading

Loading

Related research domains (42)

Ising model

The Ising model (ˈiːzɪŋ) (or Lenz-Ising model or Ising-Lenz model), named after the physicists Ernst Ising and Wilhelm Lenz, is a mathematical model of ferromagnetism in statistical mechanics. The m

Stochastic process

In probability theory and related fields, a stochastic (stəˈkæstɪk) or random process is a mathematical object usually defined as a sequence of random variables, where the index of the sequence has

White noise

In signal processing, white noise is a random signal having equal intensity at different frequencies, giving it a constant power spectral density. The term is used, with this or similar meanings, i

Units doing similar research (106)

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, our understanding of these models, in particular of what happens during their training, remains very limited. This PhD started with the introduction of the Neural Tangent Kernel (NTK) to describe the evolution of the function represented by the network during training. In the infinite-width limit, i.e. when the number of neurons in the layers of the network grows to infinity, the NTK converges to a deterministic and time-independent limit, leading to a simple yet complete description of the dynamics of infinitely-wide DNNs. This allowed one to give the first general proof of convergence of DNNs to a global minimum, and yielded the first description of the limiting spectrum of the Hessian of the loss surface of DNNs throughout training.More importantly, the NTK plays a crucial role in describing the generalization abilities of DNNs, i.e. the performance of the trained network on unseen data. The NTK analysis uncovered a direct link between the function learned by infinitely wide DNNs and Kernel Ridge Regression predictors, whose generalization properties are studied in this thesis using tools of random matrix theory. Our analysis of KRR reveals the importance of the eigendecomposition of the NTK, which is affected by a number of architectural choices. In very deep networks, an ordered regime and a chaotic regime appear, determined by the choice of non-linearity and the balance between the weights and bias parameters; these two phases are characterized by different speeds of decay of the eigenvalues of the NTK, leading to a tradeoff between convergence speed and generalization. In practical contexts such as Generative Adversarial Networks or Topology Optimization, the network architecture can be chosen to guarantee certain properties of the NTK and its spectrum.These results give an almost complete description DNNs in this infinite-width limit. It is then natural to wonder how it extends to finite-width networks used in practice. In the so-called NTK regime, the discrepancy between finite- and infinite-widths DNNs is mainly a result of the variance w.r.t. to the sampling of the parameters, as shown empirically and mathematically relying on the similarity between DNNs and random feature models.In contrast to the NTK regime, where the NTK remains constant during training, there exist so-called active regimes, where the evolution of the NTK is significant, which appear in a number of settings. One such regime appears in Deep Linear Networks with a very small initialization, where the training dynamics approach a sequence of saddle-points, representing linear maps of increasing rank, leading to a low-rank bias which is absent in the NTK regime.

Critical statistical mechanics and Conformal Field Theory (CFT) are conjecturally connected since the seminal work of Beliavin et al. (Nucl Phys B 241(2):333-380, 1984). Both exhibit exactly solvable structures in two dimensions. A long-standing question (Itoyama and Thacker in Phys Rev Lett 58:1395-1398, 1987) concerns whether there is a direct link between these structures, that is, whether the Virasoro algebra representations of CFT, the distinctive feature of CFT in two dimensions, can be found within lattice models of statistical mechanics. We give a positive answer to this question for the discrete Gaussian free field and for the Ising model, by connecting the structures of discrete complex analysis in the lattice models with the Virasoro symmetry that is expected to describe their scaling limits. This allows for a tight connection of a number of objects from the lattice model world and the field theory one. In particular, our results link the CFT local fields with lattice local fields introduced in Gheissari et al. (Commun Math Phys 367(3):771-833, 2019) and the probabilistic formulation of the lattice model with the continuum correlation functions. Our construction is a decisive step towards establishing the conjectured correspondence between the correlation functions of the CFT fields and those of the lattice local fields. In particular, together with the upcoming (Chelkak et al. in preparation), our construction will complete the picture initiated in Hongler and Smirnov (Acta Math 211:191-225, 2013), Hongler (Conformal invariance of ising model correlations, 2012) and Chelkak et al. (Annals Math 181(3):1087-1138, 2015), where a number of conjectures relating specific Ising lattice fields and CFT correlations were proven.

Franck Raymond Gabriel, Clément Hongler

The Neural Tangent Kernel is a new way to understand the gradient descent in deep neural networks, connecting them with kernel methods. In this talk, I'll introduce this formalism and give a number of results on the Neural Tangent Kernel and explain how they give us insight into the dynamics of neural networks during training and into their generalization features.