Why is Bahdanau attention called additive attention?

According to Vizuara's Lecture 3, the name comes from the scoring function: the weighted decoder state (W1·s) and the weighted encoder state (W2·h) are summed before tanh and the dot product with v. That addition of the two projected states is what makes it additive, in contrast with the dot-product scoring that Luong proposed in 2015.

Bahdanau attention is named after the surname of the first author of the 2014 paper.
The instructor recalls the paper as having roughly 44,000 to 45,000 citations.

▸In their words

“So this is addition, so this addition is happening.” Instructor · at 31:46 — open in the transcript →

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-08). Read the whole episode summary →

▸More from this episode

Q1How does Bahdanau additive attention compute alignment scores, according to Vizuara?Q3What problem did Bahdanau attention solve in RNN sequence-to-sequence translation?Q4What is the difference between attention scores and attention weights in Bahdanau attention?Q5How do you read an attention weights matrix in neural machine translation, according to Vizuara?Q6How does a Bahdanau attention decoder predict the next token, as Vizuara explains it?Q7How does additive attention differ from Luong dot-product and scaled dot-product attention?

Transcribe your links. Paste a YouTube, Google Drive or podcast link and WhipScribe gives you the transcript. Transcribe a link →