How do you read an attention weights matrix in neural machine translation, according to Vizuara?

The Vizuara instructor reads the matrix row by row: each row belongs to one output word and its cells, which sum to one, show how much that word attends to each input word. The brightest cells reveal which words are translations of each other. The matrix is square only when input and output lengths match; cross-sentence attention is generally rectangular, unlike self-attention, which is always square.

• In the 'the cat sat today' example the cell linking chat and cat is the largest in its row.
• For 'the European Economic Area' translated as 'la zone économique européenne', the strong cells run off the main diagonal because French reorders the words; European-to-européenne scores 0.83.
• The instructor says this matrix reappears throughout transformer architectures.

▸In their words

“Are forever going to see as long as you are working with transformer architecture.” Instructor · at 25:48 — open in the transcript →

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-08). Read the whole episode summary →

▸More from this episode