Google Open-Sources TorchTPU to Bring PyTorch Workloads to Its AI Chips

TL;DR

• Google released TorchTPU as an open-source project for running PyTorch workloads on its Tensor Processing Units.
• The company says developers can move existing applications to TPUs through a configuration change without rewriting them.
• The launch follows Amazon’s TorchNeuron release for its Trainium chips.
• Google worked with major AI software developers to support compatibility and introduced tools to assist migration.

Google Targets Easier Access to TPU Hardware

Google announced Tuesday that TorchTPU is now open source, giving developers another route to run PyTorch-based AI applications on its Tensor Processing Units.

The release aims to make Google’s hardware more accessible to developers whose software was built around Nvidia GPUs.

According to Google, existing PyTorch applications can move to TPUs through a configuration change without requiring developers to rewrite their code. That could lower a significant barrier to adopting Google Cloud’s AI infrastructure.

The announcement forms part of a broader effort by cloud providers to make their own chips practical alternatives for training and running AI models.

PyTorch Compatibility Is Central to Hardware Competition

PyTorch is a widely used open-source framework for building, training and deploying AI models.

Developed at Facebook, now Meta, it became closely associated with Nvidia GPUs, which were already established in AI research. Meta transferred PyTorch to the Linux Foundation in 2022.

For competing hardware providers, supporting familiar PyTorch workflows can be as important as offering powerful chips. Developers may be reluctant to switch infrastructure if doing so requires extensive engineering work or disrupts access to established tools.

TorchTPU addresses that software barrier by bringing PyTorch workloads to Google’s TPU ecosystem.

Amazon Pursues a Similar Strategy With TorchNeuron

Google’s announcement follows Amazon’s Monday release of TorchNeuron, a tool designed to move PyTorch applications onto AWS Trainium hardware without rewriting them.

Amazon also said it is contributing code to the main PyTorch project to reduce hardware-specific dependencies and make features more accessible across different processors.

Both companies are seeking to reduce the effort required to use alternatives to Nvidia.

Their approaches could give developers more infrastructure choices, but easier migration is only one consideration. Performance, cost, reliability and compatibility across real workloads will also influence adoption.

Google Adds Ecosystem Support and Migration Tools

Google said it collaborated with Meta, Hugging Face, Inferact, RadixArk and other organizations to improve TorchTPU’s compatibility with widely used AI software.

Those relationships include work involving vLLM and SGLang, tools used to serve AI models.

Google also introduced the TPU agent suite, designed to help developers move applications to its hardware and resolve errors during migration.

Separately, it updated LiteRT, its runtime for AI on phones, laptops and embedded devices, to work better with PyTorch applications.

Together, these efforts extend beyond chip compatibility to address the surrounding development and deployment workflow.

Nvidia Faces Stronger Software Competition

TorchTPU and TorchNeuron challenge an important part of Nvidia’s position: the convenience of building within a mature software ecosystem.

Making alternative hardware easier to use could encourage developers to evaluate TPUs and Trainium for more workloads.

However, the availability of migration tools does not establish that every application will transfer seamlessly or perform better after moving. Developers will still need to assess their own systems.

The launches nevertheless signal that Google and Amazon are competing on both hardware and the software needed to make that hardware useful.