🔍 Read the full analysis: How Hugging Face ❤️ TorchTPU Makes AI Work Natively On TPUs on ThorstenMeyerAI.com
TL;DR
Hugging Face announced TorchTPU, a PyTorch backend that lets supported workflows in Transformers, Diffusers and Kernels use Google TPUs through familiar device-selection APIs. The company reports large decoding speedups for two models on a specified TPU setup, but the announcement does not provide full benchmark methods, broad compatibility details or independent results.
Hugging Face has announced TorchTPU, a PyTorch backend that lets supported workflows in its Transformers, Diffusers and Kernels libraries run on Google Tensor Processing Units, as described in the original analysis. The release offers familiar ways to select a TPU device, while the company’s performance figures are limited to specified models and configurations and have not been independently verified in the supplied material.
Hugging Face presents TorchTPU as a way to direct models and tensors to a TPU through existing PyTorch library interfaces. Its examples load a Transformers model with device_map="tpu" and move a Diffusers pipeline with pipe.to("tpu"). Compiled execution is available as an option. The announcement also describes Pallas kernels tuned for TPUs, distributed training support and integration with torch.compile.
For larger models, the company says loading can split them across chips using a TensorParallelConfig. Its installation instructions target a TPU virtual machine and call for a CPU PyTorch wheel rather than a CUDA build. Hugging Face also says its Optimum TPU package is no longer required to run Transformers and Diffusers models using the new native support.
Hugging Face reports that selected kernels from its Kernels Hub and Tokamax improve decoding at larger batch sizes. On a TPU v6e-8 at batch size 64, it reports decoding 21.5 times faster for Qwen3-30B-A3B and 12.8 times faster for Gemma 4 26B-A4B, compared with each model’s default implementation. The announcement also includes example image-generation timings on a TPU v6e-4. These are company-reported results; the supplied material does not include a full account of measurement methods or independent comparisons.
A More Direct Route to TPU Capacity
TorchTPU could make Google’s TPU hardware easier to evaluate for teams whose existing models and development workflows use PyTorch. If supported tasks can run through familiar Hugging Face APIs, developers may have less code to adapt or maintain when testing TPU capacity alongside other accelerators. That could lower a practical barrier to experimenting with a different hardware platform.
The announcement does not establish that the change will improve performance or reduce costs across deployments. Results depend on the model, supported operations, batch size, TPU generation and software setup. A simpler device-selection step is useful only where the required workload is compatible and operationally reliable. The reported kernel results are specific to two named models at one stated batch size and should not be treated as a general performance guarantee.
As an affiliate, we earn on qualifying purchases.
From PyTorch/XLA to TorchTPU
Before this release, Hugging Face says PyTorch users generally ran TPU workloads through PyTorch/XLA, also known as torch_xla. Hugging Face describes that approach as functional but says lazy tensor execution, graph recompilations after code changes and more difficult debugging could make development harder. Those points are the company’s stated rationale for providing a native backend, not an independent assessment of every PyTorch/XLA workflow.
Google TPUs are specialized accelerators available through Google Cloud, ranging from individual chips to larger pods. Hugging Face previously offered Optimum TPU for inference and training support for its models. The company says TorchTPU brings TPU device support directly to Transformers and Diffusers workflows, but its announcement does not say that every model or operation in those libraries is covered.
““With TorchTPU, a native PyTorch backend, TPU is now just another device.””
— Hugging Face
Hugging Face Transformers model hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Compatibility and Results Need More Detail
The announcement does not provide a full compatibility list identifying supported models, operators or workloads, or naming unsupported ones. It also does not include independent benchmark results, detailed measurement procedures or a cost comparison for the reported speedups. The figures may vary with model versions, batch sizes, TPU generations and software configurations.
Hugging Face says it is running the Transformers test suite against TPU support and working toward nightly TPU continuous-integration testing. The supplied announcement does not say when that testing will be available, what results will be published or how quickly coverage will expand. It remains unclear how well TorchTPU will perform across a broader range of real-world training and inference workloads.
TPU acceleration for Diffusers models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
More TPU Testing Is Planned
Hugging Face says its next work includes expanding TPU validation through the Transformers test suite and adding nightly TPU testing. The company has not given a timeline or stated which compatibility and benchmark details it will publish. Developers can consult its Diffusers TPU guide for distributed and compiled use, and follow the announced installation path on TPU virtual machines.
As support develops, the most useful evidence will be a clearer account of covered models and operations, repeatable benchmarks across representative workloads, and results from ongoing test runs. Until then, teams considering deployment will need to check whether their specific workflows are supported and measure performance on their own configurations.
Tensor Processing Unit AI hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is TorchTPU?
TorchTPU is a PyTorch backend that Hugging Face says lets supported PyTorch workflows use Google TPUs as a device. The announcement describes its use with Transformers, Diffusers and Kernels.
Which Hugging Face libraries are named?
Hugging Face names Transformers, Diffusers and Kernels. The announcement does not establish that every model, operation or workflow in those libraries is supported.
Does TorchTPU replace PyTorch/XLA?
Hugging Face presents TorchTPU as a native PyTorch backend and says its Optimum TPU package is no longer required for Transformers and Diffusers models using the new support. The announcement does not describe TorchTPU as replacing every PyTorch/XLA use case.
Are the reported speedups independently verified?
No independent verification is included in the supplied announcement. Hugging Face reports speedups for two models on a TPU v6e-8 at batch size 64, compared with each model’s default implementation, but does not provide full benchmark methods or results across other workloads.
What should developers check before using it?
Developers should confirm that their model and operations are supported, then test performance and reliability on their intended TPU setup. Hugging Face says compatibility testing and nightly TPU CI work are in progress.
Primary source: Hugging Face · via ThorstenMeyerAI.com
