Publications associées à Extrapolation for Large-batch Training in Deep Learning

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Extrapolation for Large-batch Training in Deep Learning

Graph Chatbot

Chattez avec Graph Search

Towards Understanding Sharpness-Aware Minimization

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Masked Training of Neural Networks with Partial Gradients

Momentum-Based Policy Gradient with Second-Order Information

ADAGRAD Avoids Saddle Points

Metrizing Fairness

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Disparity Between Batches as a Signal for Early Stopping

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

ADAGRAD Avoids Saddle Points

Momentum-Based Policy Gradient with Second-Order Information

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Metrizing Fairness

Disparity Between Batches as a Signal for Early Stopping

Towards Understanding Sharpness-Aware Minimization

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Masked Training of Neural Networks with Partial Gradients