Variational Auto-Encoders and NVIB

This lecture covers Variational Auto-Encoders (VAE) and Nonparametric Variational Information Bottleneck (NVIB). It explains the Bayesian approach to auto-encoders, the reparameterization trick, and the taxonomy of generative models. The lecture delves into the intractability of data likelihood in VAEs, the concept of variational inference, and the use of attention-based latent spaces in Transformers. It also discusses the importance of inducing compressed representations in text data and the benefits of NVIB in generating smooth representation spaces. The presentation concludes with insights on why Transformers are effective models for language processing.