How many types of attention mechanisms are there, according to the Vizuara attention lectures?

In the opening lecture of The Attention Family, the Vizuara instructor says there are easily more than 50 reasonably famous attention types introduced by different papers, and shows a slide of roughly 35 to 37 of them, including flash, paged, sliding window, cross, bidirectional, linear, Bahdanau, deformable, ring, grouped query, multi-head latent and Kimi Delta attention. He expects new papers to keep adding more.

• The series is named The Attention Family to put all of these variants under one roof.
• The instructor's advice is to learn the categories rather than the names, so an unfamiliar variant can be slotted in on sight.

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-01). Read the whole episode summary →

▸More from this episode