Diffusion Models and Robustness

This lecture covers the mathematics behind diffusion models and their application in addressing the difficulties of training Generative Adversarial Networks (GANs). It delves into the challenges of GAN training, the formulation of Wasserstein GANs, and the application of diffusion models to improve robustness. The instructor discusses the theoretical foundations of score-based generative models using Stochastic Differential Equations (SDE) and explores the concepts of perturbation stability, Lipschitz constant estimation, and lazy training regimes in deep learning. The lecture concludes with insights on the impact of neural network architecture choices on robustness and the trade-offs between width, depth, and initialization in achieving average-case robustness.