What problem did Bahdanau attention solve in RNN sequence-to-sequence translation?

Vizuara's instructor says RNN encoder-decoder models, used until around 2014, squeezed a whole source sentence into one final hidden state used as a constant context vector for every output word, and translation broke down beyond roughly 20 words. Bahdanau attention replaced that with a context vector recomputed at each decoding step (C1, C2, C3 and so on), so the decoder gets a fresh mixture of encoder states focused on what it needs at that moment.

• The running example translates 'the cat sat today' from English to French.
• The instructor stresses the shift from one static context vector to per-token context vectors as the lecture's key point.

▸In their words

“Which is too much pressure on one single vector.” Instructor · at 1:26 — open in the transcript →

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-08). Read the whole episode summary →

▸More from this episode