Transformer Networks: Self-Attention

In course

DEMO: eiusmod adipisicing veniam exercitation

Eu sunt esse minim cillum velit commodo in consectetur excepteur laborum culpa. Fugiat veniam tempor ad nostrud excepteur. Sit sit adipisicing anim aliquip non enim consequat excepteur sit in exercitation proident et. Eiusmod qui velit nostrud ex elit aliqua proident esse qui. Veniam commodo in proident aliqua enim ad fugiat mollit. Reprehenderit duis culpa pariatur consequat esse. Sint irure in pariatur sit aliqua duis sit do reprehenderit aliquip et ad.

Description

This lecture covers Transformer networks and self-attention layers, explaining how they map sets of inputs and the concept of multi-head attention. It delves into the process of learning weights, the importance of positional encoding, and the interpretability of the heads.

Instructor

id aliquip laboris elit

Dolore nulla ut eiusmod aute fugiat velit non quis non exercitation eu mollit culpa. Aute dolore do nostrud ipsum sunt consectetur eu cupidatat mollit duis ea. Excepteur fugiat ex pariatur labore ullamco laborum irure consequat minim fugiat exercitation elit.

Official source

https://mediaspace.epfl.ch/media/0_td8k8hm4

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Ontological neighbourhood

Mathematics

Mathematical logic: Set theory

Information engineering

Machine learning: Artificial neural networks

Linguistics

Theoretical linguistics: Syntax

Related lectures (45)