Google introduces TPU 8tx and TPU 8i Max for AI agents and confidential inference
Google has introduced TPU 8tx for confidential inference powered by AMD EPYC and TPU 8i Max, which pairs two 8i chips on a single package for long-context AI agent workloads.
- Google has introduced two additional 8th-generation TPU systems: TPU 8tx for confidential inference and TPU 8i Max for long-context workloads.
- TPU 8i Max pairs two 8i chips on a single package, doubling compute performance, SRAM, and HBM bandwidth.
- TPU 8tx uses 5th Gen AMD EPYC CPUs instead of Arm-based Axion host servers, alongside a Boardfly network topology.
- Google has not yet disclosed full specifications, pricing, or release dates for either system.
Table of Contents
Two new variants in the 8th-gen TPU family
Back in April, Google announced TPU 8t for AI model training and TPU 8i for inference. Now, the company is adding TPU 8tx - built on the same silicon as 8t - targeting confidential inference and hardware isolation for regulated enterprises and government agencies. Alongside it is TPU 8i Max, which pairs two 8i chips on a single package to serve long-context AI agents.
According to Google, a modular approach allows it to deliver specialized systems "in months rather than years." The company stated that TPUs currently serve nearly 100 customers, including Meta.
TPU 8i Max and the memory bandwidth challenge
At its core, 8i Max simply places two 8i chips into a single package. What is notable is that the system packs twice the silicon into the same existing rack footprint. Google says compute performance per square foot and per watt is also higher.
According to estimates in the original report, 8i Max delivers roughly 20.2 PFLOPS FP4, 768 MB of SRAM, and about 17.2 TB/s of HBM bandwidth. By comparison, the NVIDIA Rubin R200 is rated at 50 PFLOPS FP4 and 22 TB/s HBM4 per socket, while the AMD MI455X reaches 40 PFLOPS FP4. The compute gap is steadily narrowing.
Doubling bandwidth is also a way to avoid transitioning to HBM4 immediately. With massive context windows targeting 1 million tokens, the KV cache balloons, requiring every new token to reread the cache from memory. This contrasts with NVIDIA's LPX approach: concentrating bandwidth onto a single package rather than relying on dedicated decode chips and separate context storage tiers.
Google is also rolling out accompanying software. TPU Sync is an open-source model state management layer that offloads the KV cache from HBM to host DRAM and experimentally to shared storage. Google claims KV transfer latency drops by over 90%. Additionally, the Agent Compute Engine on Google Cloud suspends idle agent sessions and swaps out their working memory.
Google claims its vLLM-TorchTPU stack achieves industry-leading price-performance on InferenceX and AgentX. However, third-party benchmarks are currently available only for the previous-generation Ironwood. In SemiAnalysis's InferenceX testing in September running Qwen3.5 397B at FP8, Ironwood delivered 50.4% more tokens per dollar than the NVIDIA B200 and 96.0% more than the B300. No 8th-generation results are available yet.
TPU 8tx for confidential inference
TPU 8tx is optimized for confidential inference and small-scale training. Hardware-enforced isolation prevents snooping on sensitive data, including model weights. Data typically must be decrypted in memory during processing, which is precisely the vulnerability this hardware isolation aims to protect.
The chip targets strictly regulated enterprises and public sector agencies, particularly those deploying frontier or proprietary models. These organizations can fine-tune models with proprietary data, distill large models into smaller ones, and train agents without shuttling sensitive data between clusters. Google's Private AI Compute already runs confidential services on TPUs, making 8tx appear to be the productized version for customers.
The host servers for 8tx use 5th Gen AMD EPYC CPUs rather than the Arm-based Axion processors found in 8t. Google did not provide a reason, though its Confidential VM catalog currently lists only AMD and Intel processors, with no Arm options available.
TPU 8tx also adopts 8i's Boardfly interconnect instead of 8t's torus topology. Google claims up to 2x lower latency compared to TPU 8t, though the measurement methodology remains unspecified. With Boardfly, the maximum hop count is 7, compared to 16 in a 1024-chip configuration.
What is still missing
Google has not yet published full specification sheets, HBM capacity, pod dimensions, availability dates, pricing, or customer lists for either system. Details regarding attestation (the ability for customers to independently verify confidentiality) for 8tx are also absent. Currently, 8tx and 8i Max, along with 8i and 8t, are listed as "coming soon" on the Cloud TPU page.
The two new systems signal Google's push to deliver domain-specific silicon faster than before. However, all performance figures for 8i Max remain estimates, and there are still no independent benchmarks for the 8th generation. Readers should wait for real-world benchmark results and official availability dates before evaluating how they stack up against NVIDIA and AMD.
Source: Tom's Hardware
Reviewed by: Hero Association
