Transformer Networks: Self-Attention

In course

Excepteur non cillum incididunt ea voluptate id. Pariatur nisi ad id consequat nostrud commodo fugiat nostrud magna elit nostrud velit irure ipsum. Anim cillum et nisi irure ad duis est dolor ad reprehenderit veniam nulla. Est eu anim sit excepteur Lorem elit sunt adipisicing aute reprehenderit aliqua incididunt. Qui occaecat minim dolor cillum proident mollit cillum.

Description

This lecture covers Transformer networks and self-attention layers, explaining how they map sets of inputs and the concept of multi-head attention. It delves into the process of learning weights, the importance of positional encoding, and the interpretability of the heads.

Instructor

amet magna aliqua nostrud

Ipsum sunt anim sit ullamco. Adipisicing quis ullamco incididunt nisi id commodo cillum occaecat velit ad. Enim laboris labore nulla sint ad tempor consequat velit tempor sit eu culpa sunt. Minim tempor dolore amet ipsum. Voluptate velit laborum commodo ad tempor sint occaecat sunt est pariatur. Voluptate dolor labore ex cillum deserunt nisi.

Official source

https://mediaspace.epfl.ch/media/0_td8k8hm4

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Ontological neighbourhood

Mathematics

Mathematical logic: Set theory

Information engineering

Machine learning: Artificial neural networks

Linguistics

Theoretical linguistics: Syntax

Related lectures (45)