Learning Sparse Features: Overfitting in Neural Networks

This lecture explores how learning sparse features can lead to overfitting in neural networks. Despite theoretical expectations, empirical evidence shows that generalization is possible due to learning meaningful features. The presentation discusses the impact of feature learning versus lazy training on generalization error scaling and the smoothness of image datasets.