Gradient Descent: Proximal Operator and Step-Size Strategies

Related lectures (29)

Page 3 of 3

Covers coordinate descent, a method for optimizing functions by updating one coordinate at a time.

Explores Dirichlet series convergence properties and absolute convergence conditions, with examples and applications.

Covers proximal operators, gradient descent, optimality conditions, and convergence analysis in optimization problems.

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Discusses proximal and subgradient descent methods for optimization in machine learning.

Explores gradient descent, loss functions, and optimization techniques in neural network training.

Introduces Quasi-Newton methods for optimization, explaining their advantages over traditional approaches like Gradient Descent and Newton's Method.

Explores the optimality of convergence rates in convex minimization problems using accelerated gradient descent methods.

Explores monotone convergence, dominated convergence, and Fatou's lemma with practical examples.