Lenka Zdeborová

About
Privacy
Disclaimer

Graph Chatbot

Related publications (72)

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Florent Gérard Krzakala, Lenka Zdeborová, Adrien Vandenbroucque, Lucas Andry Clarte, Bruno Loureiro

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

ACM2024

Asymptotics of feature learning in two-layer networks after one gradient-step

Florent Gérard Krzakala, Lenka Zdeborová, Luca Pesce, Yue Lu, Bruno Loureiro, Hugo Chao Cui

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

Universality laws for Gaussian mixtures in generalized linear models

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Ludovic Théo Stephan

A recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis has led to a number of results in the context of empirical risk minimization, Bayesian uncertainty quantification, separation of kernel methods and neural ne ...

2024

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Florent Gérard Krzakala, Lenka Zdeborová, Emanuele Troiani, Luca Pesce

We investigate the training dynamics of two-layer neural networks when learning multi-index target functions. We focus on multi-pass gradient descent (GD) that reuses the batches multiple times and show that it significantly changes the conclusion about wh ...

JMLR2024

Gaussian universality of perceptrons with random labels

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Ludovic Théo Stephan

While classical in many theoretical settings-and in particular in statistical physics-inspired works-the assumption of Gaussian i.i.d. input data is often perceived as a strong limitation in the context of statistics and machine learning. In this study, we ...

Amer Physical Soc2024

On the atypical solutions of the symmetric binary perceptron

Florent Gérard Krzakala, Lenka Zdeborová

We study the random binary symmetric perceptron problem, focusing on the behavior of rare high-margin solutions. While most solutions are isolated, we demonstrate that these rare solutions are part of clusters of extensive entropy, heuristically correspond ...

Iop Publishing Ltd2024

Statistical mechanics of the maximum-average submatrix problem

Florent Gérard Krzakala, Lenka Zdeborová, Vittorio Erba

We study the maximum-average submatrix problem, wherein given an N x N matrix J, one needs to find the k x k submatrix with the largest average number of entries. We investigate the problem for random matrices J, whose entries are i.i.d. random variables, ...

Iop Publishing Ltd2024

Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective

Florent Gérard Krzakala, Lenka Zdeborová

Recent years witnessed the development of powerful generative models based on flows, diffusion, or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical anal ...

Proceedings of the National Academy of Sciences2024

Les Houches 2022 Special Issue: Editorial

Florent Gérard Krzakala, Lenka Zdeborová

IOP Publishing Ltd2024

Counting and hardness-of-finding fixed points in cellular automata on random graphs

Lenka Zdeborová, Freya Behrens

We study the fixed points of outer-totalistic cellular automata on sparse random regular graphs. These can be seen as constraint satisfaction problems, where each variable must adhere to the same local constraint, which depends solely on its state and the ...

IOP Publishing2024

Page 2 of 8

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Florent Gérard Krzakala, Lenka Zdeborová, Emanuele Troiani, Luca Pesce

JMLR2024