How does a Bahdanau attention decoder predict the next token, as Vizuara explains it?

In Vizuara's walkthrough, the decoder concatenates the current context vector with its previous state; with 768-dimensional vectors that gives 1,536 dimensions. A linear layer projects this onto the vocabulary, for example 50,000 tokens, and softmax turns the logits into probabilities that are sampled greedily or with top-k. Another layer produces the next decoder state, and the loop runs until an end-of-sequence token appears.

• The new decoder state feeds the next round of attention scores, so every step gets different weights.
• Because decoding stops at the end-of-sequence token, the output can be shorter or longer than the input sentence.

▸In their words

“Attention scores are dependent on what is your current decoder state.” Instructor · at 24:48 — open in the transcript →

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-08). Read the whole episode summary →

▸More from this episode