Publications related to Gradient descent

Predicting nonlinear optical scattering with physics-driven neural networks

Demetri Psaltis, Carlo Gigli, Ahmed Ayoub

Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...

AIP Publishing2023

Inverse Reinforcement Learning of Pedestrian-Robot Coordination

Aude Billard, David Julian Gonon

We apply inverse reinforcement learning (IRL) with a novel cost feature to the problem of robot navigation in human crowds. Consistent with prior empirical work on pedestrian behavior, the feature anticipates collisions between agents. We efficiently learn ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Scott William Pesme

In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minim ...

2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Ludovic Théo Stephan, Bruno Loureiro

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

System and method for privacy-preserving distributed training of neural network models on distributed datasets

Jean-Pierre Hubaux, Juan Ramón Troncoso-Pastoriza, Jean-Philippe Léonard Bossuat, Apostolos Pyrgelis, David Jules Froelicher, Sinem Sav

A computer-implemented method and a distributed computer system (100) for privacy- preserving distributed training of a global neural network model on distributed datasets (DS1 to DSn). The system has a plurality of data providers (DP1 to DPn) being commun ...

2022

Metrizing Fairness

Daniel Kuhn, Yves Rychener, Bahar Taskesen

We study supervised learning problems for predicting properties of individuals who belong to one of two demographic groups, and we seek predictors that are fair according to statistical parity. This means that the distributions of the predictions within th ...

2022

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Gradient descent

Predicting nonlinear optical scattering with physics-driven neural networks

Inverse Reinforcement Learning of Pedestrian-Robot Coordination

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

A multigrid method for PDE-constrained optimization with uncertain inputs

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Masked Training of Neural Networks with Partial Gradients

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

System and method for privacy-preserving distributed training of neural network models on distributed datasets

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Metrizing Fairness

Graph Chatbot

Chat with Graph Search

Predicting nonlinear optical scattering with physics-driven neural networks

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

A multigrid method for PDE-constrained optimization with uncertain inputs

Inverse Reinforcement Learning of Pedestrian-Robot Coordination

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Masked Training of Neural Networks with Partial Gradients

Metrizing Fairness

System and method for privacy-preserving distributed training of neural network models on distributed datasets

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks