Deep Learning: Dimensionality and Data Representation

This lecture explores the landscape, performance, and curse of dimensionality in deep learning, focusing on classifying data in large dimensions, the benefits of learning data representation, and the stability towards smooth deformations. It delves into the mechanisms behind deep nets' invariance towards deformations, the geometry of loss landscapes, and the phase diagram for deep learning. Additionally, it discusses the 'jamming' transition in deep learning, two limiting algorithms based on the number of parameters, and the neural tangent kernel in modern architectures.