At what sentence length does seq2seq translation quality drop without attention?

The Vizuara instructor points to a widely shared chart of translation quality against sequence length: models relying on a single context vector lose quality significantly once a sentence passes about 20 words, while models with attention keep translation quality roughly flat as length grows. That robustness to long inputs is given as the main reason attention took off.

The instructor recalls seeing versions of this graph posted widely on LinkedIn around 2022.
A rough dimension analogy: 800 dimensions spread across 8 words is about 100 per word, across 40 words about 20 per word.

▸In their words

“The quality diminishes significantly once the word count in the sentence increases beyond 20.” Instructor · at 22:35 — open in the transcript →

Source: our own transcription of the episode. The full conversation is on The attention family (2026-09-04). Read the whole episode summary →

▸More from this episode

Q1Why is a single context vector a bottleneck in seq2seq, according to Vizuara?Q2How does an RNN encoder build the context vector in neural machine translation?Q3What does the decoder rely on in a pre-attention seq2seq translation model?Q4Why do RNN seq2seq models struggle with long-range dependency, according to Vizuara?Q6How does attention fix the context vector bottleneck in machine translation?Q7When were Bahdanau, dot-product and scaled dot-product attention introduced?

Transcribe your links. Paste a YouTube, Google Drive or podcast link and WhipScribe gives you the transcript. Transcribe a link →