Named Entity Recognition: Applications and Techniques

This lecture covers Named Entity Recognition (NER), a task focused on identifying and classifying names of people, organizations, places, and more in documents. It explores the uses of NER, such as indexing and sentiment attribution, along with commercial tools like Reuters' OpenCalais and Python libraries like NLTK NER and Spacy. The lecture delves into NER as both a sequence labeling and classification task, discussing features used in NER like part-of-speech tags and word shapes. It also examines the importance of context in NER and introduces techniques such as Generative Probabilistic Models and Hidden Markov Models. The Viterbi algorithm for sequence tagging is explained, highlighting its dynamic programming approach. The lecture concludes with a quiz on the Viterbi algorithm and the suitability of HMM models for different tasks.