Google will run ordinary PyTorch on its AI chips. The 2-cent figure is a benchmark

Google said on 20 October that PyTorch can now treat a TPU as an ordinary device. The project is called TorchTPU. Tensors sit on a device named tpu, with no special wrapper. Code can run immediately or be compiled. The collective operations that split a job across chips go through the same PyTorch tools a developer already uses for other hardware. On top of that, Google shipped native PyTorch backends for vLLM, with Inferact, and for SGLang, with RadixArk. Those are the engines people use to serve a model: caching a repeated prompt, calling a tool, splitting a long context, and running the first pass of a reply on one machine and the rest on another. The repositories are public.

The number attached to the post is 2.01 cents per million tokens. Google says that is the lowest cost in a public SemiAnalysis InferenceX comparison, for Qwen 3.5 with 397 billion parameters, of which about 17 billion are active, in FP8, on its Ironwood TPU. The comparison is a benchmark among the runs listed on that chart. It is not a rate card. A lab that wants the same figure has to match the model, the precision, the length of the prompt and the reply, and the way the cost is counted. Google does not say that is what it charges.

A familiar tool is not a spare chip

The point of the release is the software, not a new processor. Developers who write PyTorch have had a shorter path onto Nvidia’s chips than onto Google’s. TorchTPU is an attempt to close that gap so a model can be brought across without a rewrite. The PyTorch project’s own note, the same day, calls the open-source release a start. The next six to twelve months, it says, are for hardening, for more operators, and for the compile step to stop redoing its work every time a size changes. A stack that still needs a year of that work is not yet the default place a model is served.

For anyone buying inference, the useful change is that a TPU can now be driven with the same serving engines as a GPU, and that one public test put a large Qwen model at about two cents per million tokens. Both sentences are Google’s. Neither is a contract, a capacity number, or a promise that the next model will land at the same cost. The code is the thing a reader can check this week. The price is a chart.