Google Cloud on Wednesday announced its eighth generation of custom-built AI chips, known as Tensor Processing Units (TPUs). This new generation is split into two versions — one for training AI models and another for running them.
The TPU 8t is designed for AI model training, while the TPU 8i focuses on inference — the stage where AI models generate responses after users submit prompts.
Inference simply refers to how AI systems work in real-time once they are already trained.
Performance Upgrades in TPU 8
Google claims significant improvements over previous TPU generations:
- Up to 3x faster AI model training
- Around 80% better performance per dollar
- Ability to scale up to 1 million+ TPUs in a single cluster
These improvements mean businesses can access more computing power while reducing both energy consumption and overall costs. Google calls these processors TPUs instead of GPUs because they are custom-built, low-power chips specifically optimized for AI workloads.
Google Is Not Replacing Nvidia
Despite launching its own AI chips, Google is not directly competing with Nvidia.
Like other major cloud providers such as Microsoft and Amazon, Google is using its TPUs to complement Nvidia’s GPUs, not replace them.
In fact, Google confirmed that its cloud platform will support Nvidia’s latest chip, Vera Rubin, later this year.
The Bigger Picture: Future of AI Chips
In the long run, hyperscalers like Google, Amazon, and Microsoft may reduce their dependence on Nvidia by building and scaling their own AI chips.
However, at present, Nvidia continues to dominate the market.
Tech analyst Patrick Moorhead previously predicted that Google’s TPUs could hurt Nvidia back in 2016. But today, Nvidia has grown into a nearly $5 trillion company, proving that assumption wrong.
Nvidia Still Benefits from Google’s Growth
If current trends continue, Google’s expansion in AI cloud services may actually increase demand for Nvidia chips rather than reduce it.
Google also announced a collaboration with Nvidia to improve cloud performance through advanced networking.
Both companies are working on Falcon, a software-based networking technology developed by Google and open-sourced in 2023 under the Open Compute Project.
