Lecture
Optimization Trade-offs: Variance Reduction and Statistical Dimension
Related lectures (32)
Optimization Methods: Convergence and Trade-offs
Covers optimization methods, convergence guarantees, trade-offs, and variance reduction techniques in numerical optimization.
Primal-dual Optimization: Extra-Gradient Method
Explores the Extra-Gradient method for Primal-dual optimization, covering nonconvex-concave problems, convergence rates, and practical performance.
Optimality of Convergence Rates: Accelerated Gradient Descent
Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.
Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent
Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.
Primal-dual Optimization: Fundamentals
Explores primal-dual optimization, minimax problems, and gradient descent-ascent methods for optimization algorithms.
Deep Learning Building Blocks
Covers tensors, loss functions, autograd, and convolutional layers in deep learning.
Gradient Descent Methods: Theory and Computation
Explores gradient descent methods for smooth convex and non-convex problems, covering iterative strategies, convergence rates, and challenges in optimization.
Proximal Operators: Optimization Methods
Explores proximal operators, subgradient methods, and composite minimization in optimization.
Composite Convex Minimization
Covers solution methods for composite convex minimization and explores examples like ₁-regularized least squares and phase retrieval.
Double Descent Curves: Overparametrization
Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.