Publications related to Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

Michaël Unser, Pakshal Narendra Bohra

We present a statistical framework to benchmark the performance of reconstruction algorithms for linear inverse problems, in particular, neural-network-based methods that require large quantities of training data. We generate synthetic signals as realizati ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

Predicting nonlinear optical scattering with physics-driven neural networks

Demetri Psaltis, Carlo Gigli, Ahmed Ayoub

Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...

AIP Publishing2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins.All these successful methods depend on a gradient-based learning algorithm to train a model on massive a ...

EPFL2023

Penalising the biases in norm regularisation enforces sparsity

Nicolas Henri Bernard Flammarion, Etienne Patrice Boursier

Controlling the parameters' norm often yields good generalisation when training neural networks. Beyond simple intuitions, the relation between parameters' norm and obtained estimators theoretically remains misunderstood. For one hidden ReLU layer networks ...

2023

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu, Yongtao Wu

Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds. The study on NTK has been devoted to typical neural network architectures, but it is incomplete for neural networks with Hadamar ...

2022

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Deep Learning Generalization with Limited and Noisy Labels

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

Predicting nonlinear optical scattering with physics-driven neural networks

An exact mapping from ReLU networks to spiking neural networks

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Transformer Models for Vision

Penalising the biases in norm regularisation enforces sparsity

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Deep Learning Generalization with Limited and Noisy Labels

Penalising the biases in norm regularisation enforces sparsity

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Predicting nonlinear optical scattering with physics-driven neural networks

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Transformer Models for Vision

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

An exact mapping from ReLU networks to spiking neural networks