Publications de Bruno Loureiro

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs

This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension d and the number of data points n diverge with a fixed ratio α = n/d. We introduce a tractable mathematical mod ...

2025

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

Florent Gérard Krzakala, Lenka Zdeborová, Adrien Vandenbroucque, Lucas Andry Clarte, Bruno Loureiro

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic descripti ...

ACM2024

Asymptotics of feature learning in two-layer networks after one gradient-step

Florent Gérard Krzakala, Lenka Zdeborová, Luca Pesce, Yue Lu, Bruno Loureiro, Hugo Chao Cui

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

Universality laws for Gaussian mixtures in generalized linear models

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Ludovic Théo Stephan

A recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis has led to a number of results in the context of empirical risk minimization, Bayesian uncertainty quantification, separation of kernel methods and neural ne ...

2024

Gaussian universality of perceptrons with random labels

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Ludovic Théo Stephan

While classical in many theoretical settings-and in particular in statistical physics-inspired works-the assumption of Gaussian i.i.d. input data is often perceived as a strong limitation in the context of statistics and machine learning. In this study, we ...

Amer Physical Soc2024

How Two-Layer Neural Networks Learn, One (Giant) Step at a Time

Florent Gérard Krzakala, Luca Pesce, Bruno Loureiro, Ludovic Théo Stephan

For high-dimensional Gaussian data, we investigate theoretically how the features of a two-layer neural network adapt to the structure of the target function through a few large batch gradient descent steps, leading to an improvement in the approximation c ...

2024

Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs

Florent Gérard Krzakala, Luca Pesce, Bruno Loureiro, Ludovic Théo Stephan

We study the impact of the batch

n_b

on the iteration time

T

of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates. We characterize the optimal batch size minimizin ...

JMLR2024

Fluctuations, bias, variance and ensemble of learners: exact asymptotics for convex losses in high-dimension

Florent Gérard Krzakala, Bruno Loureiro

From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered by the different sources of randomness in prediction is therefore key to unde ...

Iop Publishing Ltd2023

On double-descent in uncertainty quantification in overparametrized models

Florent Gérard Krzakala, Lenka Zdeborová, Lucas Andry Clarte, Bruno Loureiro, Bruno Loureiro

Uncertainty quantification is a central challenge in reliable and trustworthy machine learning. Naive measures such as last-layer scores are well-known to yield overconfident estimates in the context of overparametrized neural networks. Several methods, ra ...

PMLR Proceedings of Machine Learning Research2023

Expectation consistency for calibration of neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Lucas Andry Clarte, Bruno Loureiro, Bruno Loureiro

Despite their incredible performance, it is well reported that deep neural networks tend to be overoptimistic about their prediction confidence. Finding effective and efficient calibration methods for neural networks is therefore an important endeavour tow ...

2023