a new way to: inference. pre-training. post-training. quant. kerneling where no one has kerneled before.
Ottawa
Joined October 2025
- Tweets1.6K
- Following24
- Followers332
- Likes662
BackboardIO retweeted
Why I care about inference efficiency:
Our best Gemma 4 12B throughput gain over vLLM was 34%.
At the same hourly GPU price, that implies about 25% lower GPU cost per output token, before Backboard fees.
For a provider with demand waiting, that extra capacity matters.
backboard.io/blog/gemma-4-12…
BackboardIO retweeted
Outperforming DeepInfer In Decode And Prefill
At 65,536 tokens per step, Backboard delivers 1.37× the prefill throughput of our DeepInfer run.
@SemiAnalysis_ @ArtificialAnlys @lm_zheng
backboard.io/blog/deepseek-v…
BackboardIO retweeted
Gemma 4 12B: 104/104 wins against vLLM. 104/104 against SGLang in our InferenceX tests.
Every measured metric. Every tested load. One RTX PRO 6000 Blackwell. All engines running BF16.
@GoogleDeepMind @NVIDIAAI @BackboardIO
BackboardIO retweeted
By every metric (208 of them) @BackboardIO beat vLLM and SGLang on InfetenceX by @SemiAnalysis_ by 1.5X on @GoogleDeepMind Gemma4. More to come.
BackboardIO retweeted
Do you like money?
At 16 concurrent AgentX sessions, Backboard delivers 26% more output throughput while making the experience dramatically faster.
For a provider running 1,000 H100s, that's potentially $4.6M in annual net capacity value, assuming $4/GPU-hour and full utilization.
No additional GPUs. Just better economics from the infrastructure they already have.
And we're doing it at full BF16 precision.
Backboard delivered up to 12.8× better p90 interactivity in AgentX than SGLang.
In @SemiAnalysis_' agentic inference benchmark, Backboard running at BF16 (full precision) achieved not just that superior interactivity but 13.6× faster time to first token. and 1.3× higher total throughput than SGLang’s published FP8 results at 16 concurrent sessions on 8× H100 GPUs.
We’ve published the full results, including the data at all concurrency levels: ow.ly/hpgB510515m
BackboardIO retweeted
We're so bad at comms, but we're getting better. dev.to/jon_at_backboardio/we…
BackboardIO retweeted
We beat DeepInfer head to head, more to come, but hope @SemiAnalysis_ notices @BackboardIO soon, even though we’re a Canadian lab without hundreds of millions of dollars and thousands of contributors. A win is a win.
RT @RobImbeault: Fresh out of the oven. We were told to use the AgentX by @SemiAnalysis_ benchmark so we ran it against the current best s…
BackboardIO retweeted
What happens when the workload keeps coming?
In our GLM 5.3 sustained test at 32 concurrent users:
@BackboardIO: 370 output tokens/sec
vLLM: 74.9
4.94X aggregate output throughput.
Full model. No Quant!
BackboardIO retweeted
Our latest GLM 5.3 full-model BF16 test:
@BackboardIO: 165.5 output tokens/sec
vLLM: 63.5 output tokens/sec
2.61X aggregate output throughput.
Same 96 workflows. Same request bodies. 32 concurrent users.
BackboardIO retweeted
Gemini 3.8 is now on Backboard Studio.
BackboardIO retweeted
Fable 5.1 is now on Backboard Studio and the Backboard Platform.
It's priced the same as Fable 5, with 75% cheaper API cache reads. It gets a lot further into a long task before it needs your input, is better at telling you when it's stuck, and its writing style is more natural.
🤖 Made with AI
BackboardIO retweeted
Backboard Vlog: Miami Company Offsite
At the start of the summer, @RobImbeault made us a promise. If we built the best coding agent in the world, he would book a private jet to a city of our choice.
We did, and we chose Miami.
@BackboardIO
Filmed & Edited by @0xseifo
BackboardIO retweeted
We beat Claude Code using Opus, at half the cost.
$280.72 vs. $552.67
github.com/harbor-framework/…
🤖 Made with AI
BackboardIO retweeted
84.3% on Terminal Bench 2.1, your code never leaves your machine, and the whole thing runs locally on desktop.
People are loving it.
@BackboardIO 👇
backboard.io
(4/4)
BackboardIO retweeted
The ultimate winner will be whoever cracks the automatically best open weight + cumulative learning (weights) from data that never leaves your premises
BackboardIO retweeted
Tired of downloading a new CLI every time you want to try a different AI model?
Meet @BackboardIO's R-CLI. One terminal tool to run over 17,000+ models (frontier, open-source, or local).
Clean up your terminal: backboard.io/products/cli
BackboardIO retweeted
AI Platforms make you choose: simplicity and speed, or complexity and power.
I joined
@BackboardIO
this summer to kill that tradeoff,
and that's why we built the Backboard Studio Desktop app.
One workspace: 3 views.
(1/5)