Primal-dual Optimization II

This lecture delves into primal-dual optimization methods, focusing on Lagrangian approaches. It covers the Swiss army knife of convex formulations, real-world applications, and the translation of unconstrained problems into constrained ones. Various primal-dual methods are explored, including penalty and augmented Lagrangian methods, Arrow-Hurwitz's method variants, splitting techniques, and second-order decomposition methods. The lecture also discusses the quadratic penalty and Lagrangian formulations, e-accuracy in constrained optimization, and the behavior of the AL dual function. Examples such as blind image deconvolution, maximum-weight cut of a graph, clustering, and neural networks are presented to illustrate the concepts.