Google launches TPU 8tx and 8i Max for secure inference

Google has announced two new systems in its eighth-generation TPU portfolio: TPU 8tx and TPU 8i Max. According to the official Google blog post, these additions mark the first time the company has introduced four TPU systems in a single year, following the earlier releases of TPU 8t and TPU 8i at Google Cloud Next in April. The primary goal of these new systems is to address specific bottlenecks in agentic AI workloads, specifically the need for hardware-level data security and the memory constraints associated with massive context windows.

Amin Vahdat, SVP and Chief Technologist for AI and Infrastructure at Google, stated that the new systems are designed to help manage complex agentic workloads by providing specialized hardware for security, density, and reasoning. The TPU business is described as a "rapidly growing business with nearly 100 customers," including frontier labs, trading firms, retailers, and universities. This expansion reflects a shift in demand from general-purpose training to specialized inference and secure execution environments for autonomous agents.

Hardware specifics and architectural changes

The announcement distinguishes between a "chip" and a "system." Google notes that while chip design is bound by multi-year silicon manufacturing lifecycles, system design—encompassing packaging, networking, and storage—allows for faster iteration to meet emerging customer needs. This modularity is central to the new TPU 8tx and TPU 8i Max configurations.

TPU 8tx: Confidential Inference TPU 8tx is built on the same silicon as TPU 8t but is paired with 5th Generation AMD EPYC CPU hosts to provide hardware-level confidential compute. The system addresses the vulnerability where data must be decrypted in memory during processing. By integrating security directly into the hardware layer, TPU 8tx keeps sensitive data and model weights protected and isolated even while in active use. It utilizes the low-latency Boardfly network created for real-time reasoning, delivering up to 2x lower latency compared to TPU 8t with a near-zero performance penalty. This configuration allows organizations to fine-tune models using private data, distill large models, and train agents through reinforcement learning without migrating sensitive data between separate clusters.

TPU 8i Max: Density and Memory Bandwidth TPU 8i Max targets the "memory wall" encountered when multiple agents work across large codebases or long video sequences. The system combines two low-latency TPU 8i chips onto a single package. This dual-chip configuration delivers double the compute, double the SRAM, and double the HBM bandwidth of the standard TPU 8i. By fitting twice the silicon within the existing rack footprint, TPU 8i Max significantly increases computing power per square foot and per watt, addressing the bottleneck where memory bandwidth limits the speed of large-scale inference.

Context: The shift to agentic infrastructure

Google’s TPU strategy has evolved from serving internal needs like voice recognition to supporting a diverse external market. The four systems in the eighth-generation portfolio now cover the entire agentic lifecycle:

• TPU 8t: Optimized for large-scale training, offering up to 2.7 times better training performance than the previous generation.
• TPU 8i: Focused on real-time reasoning, providing 80% better inference price performance than the previous generation.
• TPU 8tx: Specialized for secure, confidential inference and small-scale training in regulated environments.
• TPU 8i Max: Engineered for massive context windows and maximum compute density.

This approach contrasts with competitors who often focus on a single general-purpose accelerator. Google is leveraging its co-design partnership with Google DeepMind to create specialized hardware for specific stages of the AI workflow. The company also highlights its use of AI to build AI infrastructure, citing the use of AlphaChip’s reinforcement learning system and Gemini to accelerate chip layout processes from weeks to hours.

What it means for developers

For developers working with large language models or autonomous agents, the availability of TPU 8tx and TPU 8i Max on Google Cloud offers two distinct advantages. First, teams handling sensitive data in regulated industries (such as finance or healthcare) can now run inference and fine-tuning on hardware that maintains data isolation at the silicon level, reducing the risk of data leakage during processing. Second, developers building multi-agent systems or applications requiring very long context windows can leverage the doubled HBM bandwidth of TPU 8i Max to reduce latency caused by memory bottlenecks.

The software stack remains a key part of the value proposition. Google emphasizes a "unified, open software stack" and recent PyTorch announcements to lower the barrier to entry. This means developers can access these specialized hardware configurations without needing to write hand-tuned code for specific silicon, making TPUs more accessible as general-purpose infrastructure for AI workloads.

What to watch

• Customer adoption: With nearly 100 current customers, watch for announcements from major enterprises or government entities adopting TPU 8tx for confidential workloads.
• Performance benchmarks: Independent benchmarks comparing the 2x latency reduction of TPU 8tx and the memory throughput of TPU 8i Max against NVIDIA’s latest Hopper and Blackwell architectures will be critical for validating Google’s claims.
• Software ecosystem: The integration of these new systems with PyTorch and other open-source frameworks will determine how easily developers can migrate existing workloads to the new hardware.
• Pricing models: As Google positions TPUs as a "rapidly growing business," the pricing strategy for these specialized systems, particularly for confidential compute, will influence their competitiveness against cloud-based alternatives.