Concept
Algorithme du gradient stochastique
Publications associées (431)
On the Generalization of Stochastic Gradient Descent with Momentum
Volkan Cevher, Kimon Antonakopoulos
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...
Brookline2024Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme
In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size and momentum parameter that allows u ...
2024Understanding generalization and robustness in modern deep learning
In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...
EPFL2024Explainable Face Verification via Feature-Guided Gradient Backpropagation
Touradj Ebrahimi, Yuhang Lu, Zewei Xu
Recent years have witnessed significant advance- ment in face recognition (FR) techniques, with their applications widely spread in people’s lives and security-sensitive areas. There is a growing need for reliable interpretations of decisions of such syste ...
2024Scalable constrained optimization
Modern optimization is tasked with handling applications of increasingly large scale, chiefly due to the massive amounts of widely available data and the ever-growing reach of Machine Learning. Consequently, this area of research is under steady pressure t ...
EPFL2024