Liquid Cooling
Inference vs Training Silicon: Why They Are Parting Ways
In April 2026 Google announced its eighth-generation TPU as two chips instead of one. TPU 8t, for large-scale training, has 216 GB of HBM and 128 MB of on-chip SRAM. TPU 8i, for serving and post-training, has 288 GB of HBM and 384 MB of SRAM, with a lower peak compute rating of 10.1 PFLOPs against 12.6. These are Google's figures, and they show the shift plainly: the two jobs no longer want the same balance of memory and compute. This post…