Answered from The Attention Family: Every Attention Mechanism, Explained | New Series Launch | Lecture 1 · The attention family · 2026-09-01
The Vizuara series runs in modules of rising difficulty: foundations first (Bahdanau attention and the Attention Is All You Need paper), then position encoding, the KV cache, paged, grouped query and multi-head latent attention, then sub-quadratic designs such as sparse and Kimi Delta attention, and finally GPU kernels like flash and ring attention, Swin Transformer windowed attention and graph attention for images, video and graphs.
Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-01). Read the whole episode summary →