Attention & Transformers
Advanced1 hr
The transformer and its self-attention mechanism are the core idea behind every modern LLM. Build a visual, intuitive understanding before touching the math.
Self-attentionTransformer blocksEncoder / decoderPositional encoding
Learn from these
Attention in transformers, step-by-step
3Blue1Brown · 26 min
Transformers, the tech behind LLMs (visual intro)
3Blue1Brown · 27 min
The Illustrated Transformer
ArticleJay Alammar
Attention Is All You Need (original paper)
PaperarXiv