BREAKING NEWS🚨: Google has submitted its FIRST TPUv7 FP8 vLLM results to our Agentic Inference Benchmark and has demonstrated 1.67x better perf per dollar than even GB300 NVL72 FP4. 🚀 It is amazing to see Google go full steam ahead with externalizing the power of TPUs outside of Google.👇️ (1/4)🧵
Image from SemiAnalysis (@SemiAnalysis_)

Oct 15, 2026 · 2:36 AM UTC

68
176
36
1,570
136,508
RelevantRecentLikes
AgentX captures the four elements of an agentic workload: 🟠 Multi-turn: A session includes many user/assistant interactions (tens or hundreds) compared to a handful in a chatbot scenario. 🟠 Long context: System prompts, tool definitions, and the large number of turns make context accumulate quickly. 🟠 High prefix reuse: Since the conversation progresses linearly, the output from turn n-1 is concatenated to turn n. 🟠 Sub-agent bursts: A session launches multiple short-lived sub-agents with fresh context, which create bursty KVCache patterns. TPU has been able to demonstrate that its upcoming TorchTPU vLLM stack was able to successfully handle agentic workloads. (2/4)
Image from SemiAnalysis (@SemiAnalysis_)
3
4
73
13,861
What is surprising is that Google TorchTPU vLLM on TPUv7  FP8 with just 4 chips was able to beat GB300 NVL72 FP4 on performance per TCO across most of the pareto frontier. At low interactivity, gb300 fp4 still wins but at low interactivity tpuv7 fp8 still beats gb300 fp8. (3/4)
Image from SemiAnalysis (@SemiAnalysis_)
2
7
1
82
15,481
Great work by the Google TPU, TorchTPU, @Inferact, @RedHat, and vLLM teams on their first amazing AgentX result. We are excited to see Google submit many more models soon to demonstrate the full power of the TPU software and hardware. Stay tuned for the full article with many, many more models! Link to dashboard (4/4) inferencex.semianalysis.com/…
Qwen3.5 Inference Benchmarks | InferenceX

Qwen3.5 Inference Benchmarks | InferenceX

Compare Qwen3.5 (Qwen 3.5 397B-A17B) latency, throughput, cost and TTFT across chips and serving frameworks, from public, reproducible GitHub Actions runs.

inferencex.semianalysis.com
1
4
70
12,716
Hire @AdamHoltererer for charts
1
570
I'm hyped, TPU price wars are here
4
2,422
Bullish $GOOGL and $AVGO
3
905
vllm running a tpu is the demo i wanted
3
2,613
Is it Google's internal TPU TCO or external?
371
@Google is killing it
256
@grok is it a threat to Nvidia?
1
280
what's the latency profile look like at different batch sizes? throughput per dollar is one thing but production deployment cares more about p99 latency
1
628
mmmmm... spaghetti
Image from Adam Holter (@AdamHoltererer)
193
torch tpu finally working is the real breaking news lmao
152
1.67x on the agentic benchmark specifically is wild 👀 that's basically all my token spend now, long agent sessions running all day
143
👀
59
outsiders renting TPUs at 1.67x perf/$ is probably where nvda's inference moat starts bleeding imo
1
4
1,119
Doesn't that 1.67x figure depends on the workload mix and the pricing used for each system?
1
1,094
Google is truly control it whole tech stack from hardware all the way to the application
1
885
Google opening up TPUs like this changes the whole inference game, fr.
551
nice shot
656
This is old news yet it is meaningless as it is on qwen only and the model has some lobotomy to show this. NVDA is ahead over all aspects even DPU LPU now CPU going to be irreplaceable especially with their Vera Rubin
4
Nice to see TPUv7 + vLLM on an agentic benchmark. Multi-turn agent work grows the KV cache fast — with Galahad (beta) we keep that state on local NVMe so revisits skip recompute.
9
☝️Google pays grift fees☝️
131
Interesting👀
19
FP8 vLLM integration on TPUv7 changes the serving economics quite a bit if the custom kernels hold up outside narrow benchmarks.
695
On the supply side of externalizing TPUs: estimates compiled in early June put 2026 TPU shipments at 3.32M (DigiTimes, citing CoWoS limits) to 4.6M (BofA), on different bases. TSMC CoWoS allocation and HBM supply cap the count.
566
Is it more efficient to run DeepSeek Flash on tpus or b300s?
241
1.67x perf per dollar is great. The number I'd want next to it is perf per watt, since power is what data centers actually run short of
329
1.67x on dollars is the headline. The constraint sits one layer up: Google already locked 890 MW from Constellation reactor uprates for AI. Cheap TPU flops only ship if the megawatts do. I hold Constellation.
351
Sergey.... just sayin'...
322
what specific applications do you see benefiting most from the TPUv7 results?
337
And TPU8s started deploying in September. Google will crater token prices once they have enough of those online. Or just build exponentially chattier model, which is where Argon seems to be headed.
155
Google will be the long term AI winner. No one is positioned better.
6
1,294
is vllm hitting 1.67x on a standard 8-chip pod or does it need the full 1024-chip superpod topology to beat gb300 nvl72 on long context agentic runs
1
1
761
Google putting TPUs on an outside benchmark reads like a pitch to buyers beyond its own cloud. If TPUs start landing in third-party halls, will colocation operators plan for two accelerator families with different rack and cooling specs?
4
Incredible tpu
4
Load more replies