Generative Models: Self-supervised Learning

This lecture covers generative models, focusing on self-supervised learning and Masked Language Modelling (MLM). It explains the training inputs for BERT and the concept of prompt engineering. The lecture also delves into contrastive learning and Joint Embedding Methods, emphasizing the importance of pushing away representations of unrelated views. It introduces SimCLR, a contrastive learning framework with optimized choices, and discusses the generation of views with data augmentation. The lecture concludes with a detailed explanation of Conditional GANs and their applications in generating samples of conditional distributions.