The attention mechanism is well known for its use in Transformers. But where does it come from? It's origins lie in fixing a strange problems of RNNs.
Support me on Patreon! [ Ссылка ]
Language Modeling Playlist: [ Ссылка ]_

3blue1brown series on Transformers: [ Ссылка ]

The source code for the animations can be found here:
[ Ссылка ]

These animation in this video was made using 3blue1brown's library, manim:
[ Ссылка ]

Sources (includes the entire series): [ Ссылка ]
Chapters
0:00 Introduction
0:22 Machine Translation
2:01 Attention Mechanism
8:04 Outro

Music (In Order):
Helynt - Route 10
Helynt - Bo-Omb Battlefield
Helynt - Underwater
Philanthrope, mommy - embrace [ Ссылка ]
Helynt - Twinleaf Town

Follow me!
Website: [ Ссылка ]
Twitter: [ Ссылка ]
Github: [ Ссылка ]
Instagram: [ Ссылка ]
Patreon: [ Ссылка ]

свернуть