Lenka Zdeborová

Publications associées (72)

À propos
Confidentialité
Mentions légales

Graph Chatbot

The Gaussian equivalence of generative models for learning with shallow neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro

Understanding the impact of data structure on the computational tractability of learning is a key challenge for the theory of neural networks. Many theoretical works do not explicitly model training data, or assume that inputs are drawn component-wise inde ...

2021

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*

Florent Gérard Krzakala, Lenka Zdeborová

We analyze in a closed form the learning dynamics of the stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype ...

2021

The planted k-factor problem

Lenka Zdeborová, Gabriele Sicuro

We consider the problem of recovering an unknown k-factor, hidden in a weighted random graph. For k = 1 this is the planted matching problem, while the k = 2 case is closely related to the planted traveling salesman problem. The inference problem is solved ...

IOP PUBLISHING LTD2021

Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Gabriele Sicuro

Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. In this manuscript, we characterise the learning of a mixture of K Gaussians with generic means and covariances v ...

2021

Construction of optimal spectral methods in phase retrieval

Florent Gérard Krzakala, Lenka Zdeborová

We consider the phase retrieval problem, in which the observer wishes to recover a n-dimensional real or complex signal X⋆ from the (possibly noisy) observation of |ΦX⋆|, in which Φ is a matrix of size m×n. We consider a \emph{high-dimensional} setting whe ...

2021

Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Hugo Chao Cui

In this manuscript we consider Kernel Ridge Regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

2021

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Lenka Zdeborová, Francesca Mignacco

In this paper we investigate how gradient-based algorithms such as gradient descent (GD), (multi-pass) stochastic GD, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best general ...

2021

Learning curves of generic features maps for realistic datasets with a teacher-student model

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Hugo Chao Cui

Teacher-student models provide a framework in which the typical-case performance of high-dimensional supervised learning can be described in closed form. The assumptions of Gaussian i.i.d. input data underlying the canonical teacher-student model may, howe ...

2021

Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed

Florent Gérard Krzakala, Lenka Zdeborová, Maria Refinetti

A recent series of theoretical works showed that the dynamics of neural networks with a certain initialisation are well-captured by kernel methods. Concurrent empirical work demonstrated that kernel methods can come close to the performance of neural netwo ...

PMLR2021

Generalisation error in learning with random features and the hidden manifold model*

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro

We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features, neural networks in the lazy training regime, and the hidden manifold model. ...

IOP Publishing Ltd2021

Page 6 sur 8

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Lenka Zdeborová, Francesca Mignacco

2021

Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed

Florent Gérard Krzakala, Lenka Zdeborová, Maria Refinetti

PMLR2021

Generalisation error in learning with random features and the hidden manifold model*

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro

IOP Publishing Ltd2021