Gradient Descent: Optimization Techniques

Related lectures (28)

Page 3 of 3

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Explores optimization trade-offs, variance reduction, statistical dimension, and convergence analysis in optimization algorithms.

Explores truncated conjugate gradients for solving the trust-region subproblem in optimization on manifolds efficiently.

Covers the Newton method and its convergence properties near the optimal point.

Discusses gradient descent for optimization with equality constraints and iterative convergence criteria.

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.