Hugging Face runs Transformers and Diffusers natively on Google TPUs through TorchTPU

What Hugging Face announced

Hugging Face said on 20 October 2026 in a blog post that its main libraries now treat Google's Tensor Processing Units as a native PyTorch device. The support runs through TorchTPU, an open-source PyTorch backend for TPUs published by Google under the Apache 2.0 licence. TorchTPU ships Pallas kernels tuned for TPUs, distributed training support and a torch.compile integration, and installs on a TPU VM with one pip command.

In transformers, developers pass device_map="tpu" to from_pretrained and generate as usual. Models run in eager mode by default, with torch.compile and distributed setups as options. In Diffusers, pipe.to("tpu") moves a pipeline to the chip, and a compile call adds a graph-mode path for deployment. Large diffusion models can be sharded across chips as they load through a tensor-parallel configuration.

What Hugging Face measured

Hugging Face published sample timings on a Cloud TPU v6e-4 with each model split across four chips. A 1024 by 1024 image with 40 steps took 3.5 seconds compiled and 8.0 seconds in eager mode. A 5.2-second clip from the MiniMax-H3 video and audio model took 188 seconds compiled and 262 seconds eager. The examples include Qwen-Image-2.1, the Alibaba image model released in September under a research-only licence.

The Kernels Hub also hosts TPU kernels written in Pallas, and kernels from Google's Tokamax library can be plugged in. With a ragged-dot kernel for mixture-of-experts layers on a TPU v6e-8, Hugging Face reports decoding at batch size 64 running 21.5 times faster on Qwen3-30B-A3B and 12.8 times faster on Gemma 4 26B-A4B than the default implementation. These are vendor-run benchmarks.

What changed for developers

Until now, PyTorch users reached TPUs through PyTorch/XLA. Hugging Face describes that route as harder to debug, with lazy tensors and graph recompilations triggered by small code changes. It had built Optimum TPU to cover inference and training for its models, and says that library is no longer required for Transformers and Diffusers. The practical change is that a PyTorch codebase can move between GPU and TPU with a device string instead of a separate port.

Coverage is not complete. Hugging Face says 96.5% of the transformers tests that run on TPU pass. It has filed 34 issues on the TorchTPU repository, and 19 are fixed. The TPU runs also exposed bugs in transformers itself, including in StaticCache and the Qwen3 tensor-parallel plans, which were fixed upstream.

What to watch next

Hugging Face is adding a nightly TPU run to its transformers CI dashboard. TPU pricing is set by Google Cloud and is not stated in the announcement. Other open tool releases this autumn include GGUF support in transformers for Apple Silicon. More coverage is under Open Source AI.

Questions people ask

How do I run a Hugging Face model on a TPU?

Install TorchTPU on a TPU VM with pip install torch-tpu, then load the model in transformers with device_map="tpu" or move a Diffusers pipeline with pipe.to("tpu").

Is TorchTPU open source?

Yes. The TorchTPU repository on GitHub is under the Apache 2.0 licence, and contributions require Google's contributor licence agreement.

Does everything in transformers work on TPU yet?

Not yet. Hugging Face says 96.5% of the tests that run on TPU pass, and it is tracking the remaining failures as issues on the TorchTPU repository.

Sources

1. Hugging Face loves TorchTPU: TPUs, Natively (Hugging Face, 20 Oct 2026)
2. TorchTPU repository (Google (GitHub), 20 Oct 2026)