Language Modelling and Recurrent Neural Networks

This lecture covers the fundamentals of language modelling and recurrent neural networks (RNNs). It explains how language models predict the next word in a sequence and how RNNs address the vanishing gradient problem. The lecture introduces n-gram language models, discusses the challenges of sparsity and storage, and presents solutions like smoothing and backoff. It then delves into RNNs, explaining their architecture, training process, and the use of LSTMs to capture long-term dependencies. The lecture also explores bidirectional and multi-layer RNNs, highlighting their benefits in capturing contextual information and building complex representations.