Gradient Noise: Smooth and Nonsmooth Risks

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (29)

Page 2 of 3

Proximal Gradient Descent: Optimization Techniques in Machine Learning

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Neural Networks: Training and Optimization

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.

Optimizing Loss Functions: Gradient Descent Variants

Explores early stopping in gradient descent and variants for faster optimization.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Deep and Convolutional Networks: Generalization and Optimization

Explores deep and convolutional networks, covering generalization, optimization, and practical applications in machine learning.

Variance Reduction in Deep Learning

Explores variance reduction techniques in deep learning, covering gradient descent, stochastic gradient descent, SVRG method, and performance comparison of algorithms.

Faster Gradient Descent: Projected Optimization Techniques

Covers faster gradient descent methods and projected gradient descent for constrained optimization in machine learning.

Feed-forward Networks

Introduces feed-forward networks, covering neural network structure, training, activation functions, and optimization, with applications in forecasting and finance.

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.