Publications de Levent Dogus Sagun

Scaling description of generalization with number of parameters in deep learning

Matthieu Wyart, Clément Hongler, Mario Geiger, Stefano Spigler, Levent Dogus Sagun, Franck Raymond Gabriel, Arthur Ulysse Jacot-Guillarmod, Arthur Jacot

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

A jamming transition from under- to over-parametrization affects generalization in deep learning

Matthieu Wyart, Stefano Spigler, Levent Dogus Sagun

In this paper we first recall the recent result that in deep networks a phase transition, analogous to the jamming transition of granular media, delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. The analysis leading ...

IOP PUBLISHING LTD2019

Comparing dynamics: deep neural networks versus glassy systems

Matthieu Wyart, Mario Geiger, Marco Baity-Jesi, Stefano Spigler, Levent Dogus Sagun

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (1) the complexity of the loss landscape and of the dynamics within it, and ( ...

IOP PUBLISHING LTD2019

Jamming transition as a paradigm to understand the loss landscape of deep neural networks

Matthieu Wyart, Mario Geiger, Marco Baity-Jesi, Stefano Spigler, Levent Dogus Sagun

Deep learning has been immensely successful at a variety of tasks, ranging from classification to artificial intelligence. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understandin ...

AMER PHYSICAL SOC2019

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

Rainer Wesche, Marco Baity-Jesi, Mario Geiger, Stefano Spigler, Levent Dogus Sagun

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are the complexity of the loss-landscape and of the dynamics within it, and to wh ...

2018