Google adds on-premises TPU deployment and software tools

Google positions TPU 8tx for secure, confidential inference and says it can also support small-scale training[1]. It says TPU 8tx uses the same silicon as TPU 8t and pairs it with fifth-generation AMD EPYC CPU hosts[1]. Google reports that TPU 8tx delivers up to twice lower latency than TPU 8t, with a near-zero performance penalty[1]. TPU 8i Max is aimed at massive context windows and high compute density per rack[1]. Google says it combines two TPU 8i chips in one package and doubles TPU 8i's compute, SRAM and HBM bandwidth[1].

Google says TPU 8tx keeps sensitive data and model weights protected and isolated even while they are in active use[1]. It says organizations can use the system to fine-tune private models, distill large models and train agents with reinforcement learning without moving sensitive data between clusters[1]. For TPU 8i Max, Google says two times the silicon fits within the existing rack footprint, potentially giving data centers more computing power per square foot and per watt[1]. TechTarget reported that the system has 768 MB of on-chip static RAM and 17.2 TB/s of HBM bandwidth[2].

Google says customers can now deploy TPUs directly in their own data centers[1]. The company also says it open-sourced TorchTPU, which enables PyTorch code and Hugging Face models to run natively on TPUs[1]. Its TPU agent suite is in preview and integrates with Claude Code, Cursor and Google Antigravity, according to Google[1]. Google says synchronizing reinforcement-learning weights can cut KV-transfer latency and time to first token by more than 90%[1]. It also says its modular system approach can deliver specialized supercomputers in months rather than years[1].

TechTarget reported that Google provided no availability dates or prices for TPU 8tx and TPU 8i Max[2]. Google says this is the first year in which it has introduced four TPU systems, and that its TPU business has nearly 100 customers, including frontier labs, trading firms, retailers and universities[1]. Google identifies hardware-level data security and million-token context scale as urgent challenges, and says businesses have requested accelerators for massive context windows while highly regulated customers have sought specialized encryption[1].