Publications related to Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs

Partial discharge localization in power transformer tanks using machine learning methods

This paper presents a comparison of machine learning (ML) methods used for three-dimensional localization of partial discharges (PD) in a power transformer tank. The study examines ML and deep learning (DL) methods, ranging from support vector machines (SV ...

2024

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Pakshal Narendra Bohra

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

Rahul Parhi

We study the problem of estimating an unknown function from noisy data using shallow ReLU neural networks. The estimators we study minimize the sum of squared data-fitting errors plus a regularization term proportional to the squared Euclidean norm of the ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

Michaël Unser, Pakshal Narendra Bohra

We present a statistical framework to benchmark the performance of reconstruction algorithms for linear inverse problems, in particular, neural-network-based methods that require large quantities of training data. We generate synthetic signals as realizati ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Polynomial-time universality and limitations of deep learning

Emmanuel Abbé

The goal of this paper is to characterize function distributions that general neural networks trained by descent algorithms (GD/SGD), can or cannot learn in polytime. The results are: (1) The paradigm of general neural networks trained by SGD is poly-time ...

WILEY2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

TOPO-Loss for continuity-preserving crack detection using deep learning

Pascal Fua, Katrin Beyer, Bryan German Pantoja Rosero

We present a method for segmenting cracks in images of masonry buildings damaged by earthquakes. Existing methods of crack detection fail to preserve the continuity of cracks, and their performance deteriorates with imprecise training labels. We address th ...

2022

Multivariate time series forecasting for freeway networks

Semin Kwak

This dissertation introduces traffic forecasting methods for different network configurations and data availability.Chapter 2 focuses on single freeway cases.Although its topology is simple, the non-linearity of traffic features makes this prediction still ...

EPFL2022

Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs

Graph Chatbot

Chat with Graph Search

Partial discharge localization in power transformer tanks using machine learning methods

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

Deep Learning Generalization with Limited and Noisy Labels

Polynomial-time universality and limitations of deep learning

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

TOPO-Loss for continuity-preserving crack detection using deep learning

Multivariate time series forecasting for freeway networks

Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods

Deep Learning Generalization with Limited and Noisy Labels

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Partial discharge localization in power transformer tanks using machine learning methods

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Multivariate time series forecasting for freeway networks

Polynomial-time universality and limitations of deep learning

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

TOPO-Loss for continuity-preserving crack detection using deep learning