Lecture

Sparse Regression and Convex Optimization

L1 Regularization: Sparse Solutions and Dimensionality Reduction

Delves into L1 regularization, sparse solutions, and dimensionality reduction in the context of machine learning.

Convex Optimization: Examples of Convex Functions

Explores convex optimization, convex functions, and their properties, including strict convexity and strong convexity, as well as different types of convex functions like linear affine functions and norms.

Gradient Descent Methods

Covers gradient descent methods for convex and nonconvex problems, including smooth unconstrained convex minimization, maximum likelihood estimation, and examples like ridge regression and image classification.

From Stochastic Gradient Descent to Non-Smooth Optimization

Covers stochastic optimization, sparsity, and non-smooth minimization via subgradient descent.

Optimization Techniques: Convexity and Algorithms in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity, algorithms, and their applications in ensuring efficient convergence to global minima.

Convex Optimization Tutorial: KKT Conditions

Explores KKT conditions in convex optimization, covering dual problems, logarithmic constraints, least squares, matrix functions, and suboptimality of covering ellipsoids.

Sparse Regression

Covers the concept of sparse regression and the use of Gaussian additive noise in the context of MAP estimator and regularization.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Stochastic Gradient Descent: Optimization Techniques

Explores Stochastic Gradient Descent with Averaging, comparing it with Gradient Descent, and discusses challenges in non-convex optimization and sparse recovery techniques.

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.