How does masking attention scores with minus infinity before softmax work?

Vizuara's approach sets every upper-triangle attention score to minus infinity and only then applies softmax along each row. Since e to the minus infinity is 0, the masked positions get weight 0 and also add nothing to the softmax denominator, so future keys never influence the current weights. In Sreedath Panat's example, a row of scores 2, 3 and 5 for a first-position query becomes 2, minus infinity, minus infinity, and softmax returns 1, 0 and 0.

• The lecture walks through the sentence The cat sat on the, with mat as the next token, masking keys K2 to K5 for query Q1.
• The result is clean on both sides: no future value vectors in the weighted sum and no future keys in the weights.

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-24). Read the whole episode summary →

▸More from this episode