Google Unveils Eighth-Generation Tensor Processor Units (TPUs) for Training and Inference

Google's rate of development for their tensor processor units (TPUs) has been remarkable. The company is already preparing their eighth generation of TPUs, which were announced earlier this year.

The TPUv8 family includes both the TPU 8t for training and the TPU 8i for inference. This unique approach sets Google apart from other hyperscalers, who have focused solely on developing inference hardware while relying on commodity hardware for training.

The TPU 8i is also notable for being a major consumer of Google's in-house developed CPU, Axion, which is used in a 2-to-1 ratio inside Google's nodes. This switch to Axion has moved Google to their own hardware and Arm architecture.