Answered from Turing Award Winner: The Difference Between a CPU, GPU, and TPU | David Patterson · Ryan Peterman · 2026-11-04
Brain float 16 is the floating-point format Google introduced with the TPU, named after the Google Brain research group. David Patterson says it was the first format to give the exponent more bits than the fraction. Scientific computing mostly uses 64-bit numbers with a 50-plus-bit fraction for precision, but machine learning needs range far more than precision, so Google made that trade.
Source: our own transcription of the episode. The full conversation is on Ryan Peterman (2026-11-04). Read the whole episode summary →