Gradient Descent on Two-Layer ReLU Neural Networks

This lecture by the instructor covers the analysis of gradient descent on wide two-layer ReLU neural networks. Starting with supervised learning concepts, it delves into the gradient flow of empirical risk and the dynamics in the infinite width limit. The presentation explores global convergence, regularized and unregularized cases, and implicit regularization. Through illustrations and numerical experiments, the lecture demonstrates the implicit regularization for linear classification and neural networks. Theoretical results on statistical efficiency and two implicit regularizations in one dynamics are discussed, leading to perspectives on mathematical models for deeper networks and the quantification of convergence speed.