Publication
Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design
Related publications (112)
Generalization of Scaled Deep ResNets in the Mean-Field Regime
Volkan Cevher, Grigorios Chrysos, Fanghui Liu
Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...
2024Deep Learning Theory Through the Lens of Diagonal Linear Networks
In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of -layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...
EPFL2024Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE
Marco Picasso, Alexandre Caboussat, Maude Girardin
We present a finite elements-neural network approach for the numerical approximation of parametric partial differential equations. The algorithm generates training data from finite element simulations, and uses a data -driven (supervised) feedforward neura ...
Lausanne2024Deep Learning Generalization with Limited and Noisy Labels
Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...
EPFL2023From Kernel Methods to Neural Networks: A Unifying Variational Formulation
The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...
New York2023Supervised learning and inference of spiking neural networks with temporal coding
The way biological brains carry out advanced yet extremely energy efficient signal processing remains both fascinating and unintelligible. It is known however that at least some areas of the brain perform fast and low-cost processing relying only on a smal ...
EPFL2023Benign Overfitting in Deep Neural Networks under Lazy Training
Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu
This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...
2023Predicting nonlinear optical scattering with physics-driven neural networks
Demetri Psaltis, Carlo Gigli, Ahmed Ayoub
Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...
AIP Publishing2023