Publication

Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs

Publications associées (51)

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

, , ,

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Momentum-Based Policy Gradient with Second-Order Information

Patrick Thiran, Negar Kiyavash, Mohammadsadegh Khorasani, Saber Salehkaleybar

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

2022

The respiratory system is one of the major components of the speech production system. Any alteration in breathing can result in changes in speech. Specific breathing characteristics, such as breathing rate and tidal volume, can indicate a person's patholo ...

2021

On The Relationship Between Speech-Based Breathing Signal Prediction Evaluation Measures And Breathing Parameters Estimation

Mathew Magimai Doss, Zohreh Mostaani, Venkata Srikanth Nallanthighal

IEEE2021

Population pharmacokinetic model selection assisted by machine learning

Jan Sickmann Hesthaven, Nadia Terranova

A fit-for-purpose structural and statistical model is the first major requirement in population pharmacometric model development. In this manuscript we discuss how this complex and computationally intensive task could benefit from supervised machine learni ...

SPRINGER/PLENUM PUBLISHERS2021

Purpose: To develop a scan-specific model that estimates and corrects k-space errors made when reconstructing accelerated MRI data. Methods: Scan-specific artifact reduction in k-space (SPARK) trains a convolutional-neural-network to estimate and correct k ...

WILEY2021

Three-dimensional (3D) knee angle measurement is one of the key measures in human gait analysis. Inertial sensor capable of measuring joint motion under unconstrained conditions is a practical tool for clinical evaluation and rehabilitation. An inertial me ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

A physics-informed machine learning framework is developed for the reduced-order modeling of parametrized steady-state partial differential equations (PDEs). During the offline stage, a reduced basis is extracted from a collection of high-fidelity solution ...

2020