Publications de Leonardo Petrini

How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model

Matthieu Wyart, Umberto Maria Tomasini, Francesco Cagnetta, Alessandro Favero, Leonardo Petrini

Deep learning algorithms demonstrate a surprising ability to learn high-dimensional tasks from limited examples. This is commonly attributed to the depth of neural networks, enabling them to build a hierarchy of abstract, low-dimensional data representatio ...

American Physical Society2024

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Leonardo Petrini

Artificial intelligence, particularly the subfield of machine learning, has seen a paradigm shift towards data-driven models that learn from and adapt to data. This has resulted in unprecedented advancements in various domains such as natural language proc ...

EPFL2023

Learning sparse features can lead to overfitting in neural networks

Matthieu Wyart, Francesco Cagnetta, Leonardo Petrini

It is widely believed that the success of deep networks lies in their ability to learn a meaningful representation of the features of the data. Yet, understanding when and how this feature learning improves performance remains a challenge. For example, it ...

Iop Publishing Ltd2023

How deep convolutional neural networks lose spatial information with training

Matthieu Wyart, Umberto Maria Tomasini, Francesco Cagnetta, Leonardo Petrini

A central question of machine learning is how deep nets manage to learn tasks in high dimensions. An appealing hypothesis is that they achieve this feat by building a representation of the data where information irrelevant to the task is lost. For image da ...

Iop Publishing Ltd2023

Relative stability toward diffeomorphisms indicates performance in deep nets

Matthieu Wyart, Alessandro Favero, Mario Geiger, Leonardo Petrini

Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this qu ...

IOP Publishing Ltd2022

Geometric compression of invariant manifolds in neural networks

Matthieu Wyart, Mario Geiger, Leonardo Petrini

We study how neural networks compress uninformative input space in models where data lie in d dimensions, but the labels of which only vary within a linear manifold of dimension d(parallel to) < d. We show that for a one-hidden-layer network initialized wi ...

IOP PUBLISHING LTD2021

Relative stability toward diffeomorphisms indicates performance in deep nets

Matthieu Wyart, Alessandro Favero, Mario Geiger, Leonardo Petrini

Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this qu ...

2021

Landscape and training regimes in deep learning

Matthieu Wyart, Mario Geiger, Leonardo Petrini

Deep learning algorithms are responsible for a technological revolution in a variety oftasks including image recognition or Go playing. Yet, why they work is not understood.Ultimately, they manage to classify data lying in high dimension – a feat generical ...

2021