The Hidden Convex Optimization Landscape of Deep Neural Networks

This lecture explores the history of artificial neural networks, the impact of deep learning, the challenges in neural networks, the role of architecture, and the optimization landscape of deep neural networks. It delves into the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models. The lecture discusses the interpretability of deep learning models, the importance of convex regularizers, and the convergence of unregularized gradient flow to the optimum of the convex program. It also covers the training of ReLU and polynomial neural networks, the application of convex optimization theory, and the open problems in neural network optimization.