Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
This lecture covers the concept of attention mechanism in machine translation, addressing the bottleneck problem in sequence-to-sequence models. The instructor explains how attention provides a solution by allowing the decoder to focus on specific parts of the source sequence, improving NMT performance significantly. Attention also helps with the vanishing gradient problem and provides interpretability by showing what the decoder is focusing on. Different attention variants are discussed, including dot-product, multiplicative, and additive attention, each with its advantages. The lecture concludes with a summary of attention in NMT, highlighting its effectiveness in learning soft alignments between input and output tokens.
This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.
Watch on Mediaspace