Answered from Dot product attention and scaled dot product attention | The attention family series - Lecture 4 · The attention family · 2026-09-11
Vizuara's Lecture 4 credits scaled dot product attention to the Attention Is All You Need paper, which the instructor says has almost 260,000 citations, and plain dot product attention to Luong and co-authors in 2015. The instructor calls the scaled version by far the most popular and most frequently used scoring rule, and says nearly every attention mechanism covered later in the nine-lecture series uses it, starting with self-attention, which Lecture 5 builds by hand from scratch.
“Scale dot product mechanism is by far the most popular and most frequently used.” Instructor · at 29:28 — open in the transcript →
Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-11). Read the whole episode summary →