Publications by Arthur Ulysse Jacot-Guillarmod

Theory of Deep Learning: Neural Tangent Kernel and Beyond

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, o ...

EPFL2022

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Clément Hongler, Arthur Ulysse Jacot-Guillarmod, Franck Raymond Gabriel, Arthur Jacot

The Neural Tangent Kernel is a new way to understand the gradient descent in deep neural networks, connecting them with kernel methods. In this talk, I'll introduce this formalism and give a number of results on the Neural Tangent Kernel and explain how th ...

ASSOC COMPUTING MACHINERY2021

Implicit Regularization of Random Feature Models

Clément Hongler, Francesco Spadaro, Franck Raymond Gabriel, Berfin Simsek, Arthur Ulysse Jacot-Guillarmod

Random Feature (RF) models are used as efficient parametric approximations of kernel methods. We investigate, by means of random matrix theory, the connection between Gaussian RF models and Kernel Ridge Regression (KRR). For a Gaussian RF model with P feat ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Ulysse Jacot-Guillarmod, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Scaling description of generalization with number of parameters in deep learning

Matthieu Wyart, Clément Hongler, Mario Geiger, Stefano Spigler, Levent Dogus Sagun, Franck Raymond Gabriel, Arthur Ulysse Jacot-Guillarmod, Arthur Jacot

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

Neural Tanget Kernel: Convergence and Generalization in Neural Networks

Clément Hongler, Franck Raymond Gabriel, Arthur Ulysse Jacot-Guillarmod

At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit [12, 9], thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018