Publications related to Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

Marco Picasso, Alexandre Caboussat, Maude Girardin

We present a finite elements-neural network approach for the numerical approximation of parametric partial differential equations. The algorithm generates training data from finite element simulations, and uses a data -driven (supervised) feedforward neura ...

Lausanne2024

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Jan Sickmann Hesthaven, Federico Pichi

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real -time and many-query evaluations of parametric ...

San Diego2024

Explainable Face Verification via Feature-Guided Gradient Backpropagation

Touradj Ebrahimi, Yuhang Lu, Zewei Xu

Recent years have witnessed significant advance- ment in face recognition (FR) techniques, with their applications widely spread in people’s lives and security-sensitive areas. There is a growing need for reliable interpretations of decisions of such syste ...

2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

New York2023

Supervised learning and inference of spiking neural networks with temporal coding

Ana Stanojevic

The way biological brains carry out advanced yet extremely energy efficient signal processing remains both fascinating and unintelligible. It is known however that at least some areas of the brain perform fast and low-cost processing relying only on a smal ...

EPFL2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

Predicting nonlinear optical scattering with physics-driven neural networks

Demetri Psaltis, Carlo Gigli, Ahmed Ayoub

Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...

AIP Publishing2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Graph Chatbot

Chat with Graph Search

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Explainable Face Verification via Feature-Guided Gradient Backpropagation

Deep Learning Generalization with Limited and Noisy Labels

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Supervised learning and inference of spiking neural networks with temporal coding

Benign Overfitting in Deep Neural Networks under Lazy Training

Predicting nonlinear optical scattering with physics-driven neural networks

Generalization of Scaled Deep ResNets in the Mean-Field Regime

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Supervised learning and inference of spiking neural networks with temporal coding

Deep Learning Generalization with Limited and Noisy Labels

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

Explainable Face Verification via Feature-Guided Gradient Backpropagation

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Predicting nonlinear optical scattering with physics-driven neural networks

Benign Overfitting in Deep Neural Networks under Lazy Training