Publications de Emanuele Troiani

Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-Dimensional Tokens

Lenka Zdeborová, Emanuele Troiani, Luca Biggio, Vittorio Erba

Current progress in artificial intelligence is centered around so-called large language models that consist of neural networks processing long sequences of high-dimensional vectors called tokens. Statistical physics provides powerful tools to study the fun ...

American Physical Society2025

Gibbs sampling the posterior of neural networks

Lenka Zdeborová, Emanuele Troiani, Giovanni Piccioli

In this paper, we study sampling from a posterior derived from a neural network. We propose a new probabilistic model consisting of adding noise at every pre- and post-activation in the network, arguing that the resulting posterior can be sampled using an ...

Iop Publishing Ltd2024

The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Florent Gérard Krzakala, Lenka Zdeborová, Emanuele Troiani, Luca Pesce

We investigate the training dynamics of two-layer neural networks when learning multi-index target functions. We focus on multi-pass gradient descent (GD) that reuses the batches multiple times and show that it significantly changes the conclusion about wh ...

JMLR2024

Sparse representations, inference and learning

Emanuele Troiani

In recent years statistical physics has proven to be a valuable tool to probe into large dimensional inference problems such as the ones occurring in machine learning. Statistical physics provides analytical tools to study fundamental limitations in their ...

IOP Publishing Ltd2024

Asymptotic Characterisation of the Performance of Robust Linear Regression in the Presence of Outliers

Florent Gérard Krzakala, Emanuele Troiani, Vittorio Erba

We study robust linear regression in highdimension, when both the dimension d and the number of data points n diverge with a fixed ratio α = n/d, and study a data model that includes outliers. We provide exact asymptotics for the performances of the empiri ...

2024

Rigorous Dynamical Mean-Field Theory for Stochastic Gradient Descent Methods

Florent Gérard Krzakala, Lenka Zdeborová, Emanuele Troiani

We prove closed-form equations for the exact high-dimensional asymptotics of a family of first-order gradient-based methods, learning an estimator (e.g., M-estimator, shallow neural network) from observations on Gaussian data with empirical risk minimizati ...

Society for Industrial & Applied Mathematics (SIAM)2024

Optimal denoising of rotationally invariant rectangular matrices

Florent Gérard Krzakala, Lenka Zdeborová, Emanuele Troiani, Vittorio Erba

In this manuscript we consider denoising of large rectangular matrices: given a noisy observation of a signal matrix, what is the best way of recovering the signal matrix itself? For Gaussian noise and rotationally-invariant signal priors, we completely ch ...

2022