Self-Supervised Learning: State of the Art

This lecture covers the importance of self-supervised learning, focusing on transfer learning and pre-training tasks. It explores SSL as a prediction task, context encoders for feature learning by inpainting, and unsupervised visual representation learning. The lecture delves into tasks like predicting image rotations, imageNet linear evaluation, contrastive learning, and the choice of augmentations. It also discusses momentum contrast for unsupervised visual representation learning, BYOL approach, and what should not be contrastive in contrastive learning. The instructor emphasizes the impact of pre-training domains and the scalability of vision learners with masked autoencoders.