What made Google's TPU so much faster than GPUs, according to David Patterson?

David Patterson says Google designed the TPU from a clean slate with no graphics legacy. It centred on a giant matrix-multiply unit, dropped hardware caches because machine-learning memory accesses are predictable enough for software to schedule, and used a narrow floating-point format. The result was about 30 times better at inference than a contemporary GPU and 80 times better than a CPU.

• Patterson calls the TPU announcement the watershed moment: Intel started buying companies, NVIDIA reworked its designs for machine learning, and hyperscalers began their own chips.

▸In their words

“At the heart of neural networking is a matrix multiply.” David Patterson · at 5:22 — open in the transcript →

Source: our own transcription of the episode. The full conversation is on Ryan Peterman (2026-11-04). Read the whole episode summary →

▸More from this episode