Landscape and Generalisation in Deep Learning

This lecture explores the loss landscape and performance in deep learning, focusing on the challenges of classifying data in large dimensions and the revolution in artificial intelligence. It delves into the high-dimensional, non-convex landscape of deep learning, discussing the role of over-parametrization and the impact on generalization. The lecture also covers the quantification of fluctuations induced by initialization, the neural tangent kernel, and the hydrodynamic description of feature learning. Additionally, it analyzes the phase diagram for deep learning, the role of over-parametrization, and the performance comparison between different regimes. The key takeaway is the importance of the jamming transition and the optimal performance achieved near it.