What motivated self-attention mechanisms in transformer-based LLMs in the first place?

A made a short video covering
– the limitations of RNNs
– the original (Bahdanau) attention mechanism for RNNs,
– how it all led to the original Transformer architecture used in LLMs https://t.co/oMLNlJMef6

— Sebastian Raschka (@rasbt) 21 août 2023

What motivated self-attention mechanisms in transformer-based LLMs in the first place? A made a short video covering
– the limitations of RNNs
– the original (Bahdanau) attention mechanism for RNNs, – how it all led to the original Transformer architecture used in LLMs

→ View original post on X — @rasbt,