Google Made TPUs a Native PyTorch Device. For Most Founders, That's a Price Negotiation, Not a Migration
By Surya Pratap
October 22, 2026
8 min read
AI & Technology
October 22, 2026
8 min read
One line to move, more to be fastHover to exploreMost AI startups never think about which chip runs their model. They call an API, and the provider decides. But a growing number self-host open-weight models for cost, privacy or control — and for them, the chip has meant Nvidia, because that is where PyTorch code runs without friction. Google's release this week is an attempt to change that default.
This article is based on Google Cloud's announcement "TorchTPU is now open source: Run native PyTorch on TPUs" and on coverage by Forkast and MarkTechPost. The performance and compatibility figures are Google's and its partners', not independently verified. The reading from section 3 onward is mine.
On 20 October 2026, at PyTorch Conference, Google open-sourced TorchTPU, its native PyTorch stack for Tensor Processing Units. It was co-developed with the PyTorch team at Meta and, according to Forkast, is released under Apache 2.0.
The core idea is that a TPU becomes an ordinary PyTorch device, next to GPUs and CPUs. Code that targets "cuda" can target "tpu" instead. Google describes the workflow as: "debug your models with eager mode, then transition to production-level performance via torch.compile." Custom kernels are written in Pallas, Google's kernel language, rather than CUDA.
Much of the ecosystem a startup would actually use is wired in:
device_map="tpu"", plus Diffusers.Google and its partners reported:
Meta's Jana van Greunen called it "a massive step forward". Hugging Face's Jeff Boudier said "hardware diversity is essential to making open-source AI accessible to everyone."
All of these are vendor or partner figures. None has been independently tested.
The device swap is one line. Speed is not. Forkast's analysis is direct: "one cannot achieve peak TPU performance with a simple device swap." Specifically:
Changing the device is a line of code. Getting peak performance on a different chip is still engineering. The value is in being able to move at all.
If you only call model APIs, nothing changes for you directly. Your provider chooses its chips. At most, more competition in compute nudges API prices down over time.
If you self-host open-weight models — for cost at volume, data control, or a fine-tuned model of your own — this matters. Until now, the realistic choice was Nvidia GPUs, from a cloud or a GPU provider, at their price. A credible second accelerator that runs the same PyTorch and vLLM code changes that conversation, even if you never move.
If you train or fine-tune, TorchTitan and Hugging Face support make experimenting on TPUs cheaper to try, though large training runs are where TPU-specific tuning pays most.
Keep your code device-agnostic. Search your code for hard-coded "cuda" and replace it with a configurable device. Prefer standard PyTorch operations and well-supported libraries over hand-written CUDA unless you have measured that you need it.
Serve through portable layers. If you self-host, run inference through vLLM or SGLang and load models through Hugging Face. Those are the layers TorchTPU supports, so the same deployment can be pointed at either accelerator.
Benchmark your own workload. Run your model, your prompt lengths and your batch sizes on both, and compare cost per million tokens at the latency you need. Vendor benchmarks use their best-case models; yours is the only number that matters.
Use it in negotiation. Even a small, real benchmark on TPUs gives you a credible alternative when you discuss committed-use discounts or reserved GPU capacity. The option is worth something before you exercise it.
Portability is the same habit everywhere
Keeping model code free of one chip vendor is the hardware version of the advice we gave after the US order on DeepSeek: know what your stack depends on, and make swapping one part a configuration change, not a rebuild.
That TPUs are now faster or cheaper than GPUs for you. That depends on your model, traffic and pricing. The published figures come from Google and its partners on workloads they chose.
That PyTorch code runs unchanged at full speed. It runs, by Google's account, with minimal changes. Getting good utilisation still requires TPU-specific work.
That TPUs are easy to get. TPUs are available through Google Cloud. Availability, quotas and prices for small companies were not part of this announcement.
That Nvidia's position has changed. One open-source backend does not shift the market by itself. It lowers one switching cost, for teams that self-host.
Google and Meta have open-sourced TorchTPU, making Google's TPUs a native PyTorch device under Apache 2.0, with vLLM, SGLang, Hugging Face, TorchTitan and Ray working on top. Google reports strong utilisation and compatibility, but the figures are its own, and peak performance still requires TPU-specific engineering; custom CUDA kernels don't carry over.
For startups that only call APIs, it changes little. For those that self-host models, it creates a realistic second accelerator for the same code — which is useful even if you never switch.
If you self-host, remove hard-coded "cuda" from your code this sprint, serve through vLLM or SGLang, and run one benchmark of your own workload on TPUs. Then bring the result to your next GPU pricing conversation.
Sources: Google Cloud, "TorchTPU is now open source: Run native PyTorch on TPUs", October 2026 — the co-development with Meta, eager mode and torch.compile, Pallas kernels, the vLLM, SGLang, llm-d, Hugging Face, TorchTitan, Ray, TorchRec and VeRL integrations, the device_map="tpu" quote, the 52% MFU figure on up to 8,192 TPU v7x chips, Zenteiq's throughput figures, and the quotes from Jana van Greunen and Jeff Boudier. Forkast — the 20 October date, the Apache 2.0 licence, the "cuda" to "tpu" device change, the 98% no-crash rate across 5,000 Hugging Face models and 97% numerical parity, and the limitations: peak performance needing TPU-specific work, non-portable CUDA kernels, the single user-space process and the proprietary instruction set. MarkTechPost — coverage of the release. The reading in sections 3 to 6 is mine.
IdeaToMVP Academy
4-week live cohort for founders. Learn to ship AI agents, scope MVPs, and automate your business — taught by the same team that writes these guides.
