Primal-dual Optimization: Lagrangian Methods

This lecture covers primal-dual optimization, focusing on Lagrangian methods. It explains the Swiss army knife of convex formulations, e-accurate solutions, and various primal-dual methods like penalty, augmented Lagrangian, Arrow-Hurwitz's, and splitting techniques. The lecture also discusses quadratic penalty and Lagrangian formulations, the unification of Lagrangian and penalty approaches, and the behavior of the augmented Lagrangian dual function. It introduces the augmented Lagrangian method, its convergence, drawbacks, and enhancements, including inexact approaches for subproblems. The linearized augmented Lagrangian method and its convergence are also presented, along with an example of basis pursuit in de-noising and data compression.