podcasts / The attention family / episode / question
Question

When were Bahdanau, dot-product and scaled dot-product attention introduced?

Answered from Before Attention: The bottleneck of a single context vector | The Attention Family, Lecture 2 · The attention family · 2026-09-04

In the roadmap at the end of Vizuara's lecture, Bahdanau attention, also called additive attention, is dated to 2014, dot-product attention to 2015, and scaled dot-product attention to 2017, when it was proposed in the 'Attention Is All You Need' paper. The series covers them in that order, starting with Bahdanau attention in Lecture 3.

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-04). Read the whole episode summary →

▸More from this episode

Q1Why is a single context vector a bottleneck in seq2seq, according to Vizuara?Q2How does an RNN encoder build the context vector in neural machine translation?Q3What does the decoder rely on in a pre-attention seq2seq translation model?Q4Why do RNN seq2seq models struggle with long-range dependency, according to Vizuara?Q5At what sentence length does seq2seq translation quality drop without attention?Q6How does attention fix the context vector bottleneck in machine translation?
Transcribe your links. Paste a YouTube, Google Drive or podcast link and WhipScribe gives you the transcript. Transcribe a link →