Lexicons, n-grams and Language Models

This lecture covers the concepts of lexicons, n-grams, and language models. It starts with an introduction to lexicons and their field implementations, followed by an explanation of n-grams and probabilities. The lecture delves into the n-gram approach, smoothing techniques, and examples of language identification and spelling error correction. It emphasizes the importance of lexica for recognizing and classifying words, the effectiveness of n-grams for various tasks, and the significance of smoothing techniques in estimating n-gram probabilities.