BackboardIO

You can gauge the strength of an inference system without taking risks. We license our engine to cloud providers, inference platforms, and enterprises. It runs on your GPUs, your models, and your workload. Bring us the workload you serve today. We'll measure against your current stack, using the same rules we publish by: constant conditions, full result tables, and included failures. ow.ly/AzMW5105cVn

Output throughput is the work that counts with AI inference: it’s the tokens you actually generate with each GPU. Backboard leads at every concurrency level with Qwen in SemiAnalysis’ AgentX benchmark, with gains ranging from 1.1x to 2.2x. Responsiveness and capacity don't have to involve tradeoffs. You can get both with the right engine. ow.ly/YLeE5105cUE

Your inference engine can determine your company’s financial future. A megawatt infrastructure running open-model inference at 60% utilization can gross about $70 million per year, Dylan Patel (@dylan522p) noted at the Runtime conference. The engine becomes the margin. Backboard’s 31% throughput edge over SGLang in Qwen on the same hardware is not a rounding error, it's the profit and loss statement. That's the game we're playing. ow.ly/zRsH5105cRV

GPU capacity is expensive and difficult to secure. Learn how inference efficiency can increase AI serving capacity from the hardware you already have.

What he said 👇

You don’t have to buy more hardware to see faster inference. In @SemiAnalysis_' AgentX benchmark, Backboard produces 10,525 tokens per second for each GPU in Qwen. SGLang produces 8,013. That's 31% more work out of eight H100 GPUs at full bf16 precision against SGLang’s FP8. This performance can reduce your GPU costs and keep you running the systems you already have. ow.ly/6K2P5105cQM

We’d rather show you a narrower advantage than make you find it. In SemiAnalyisis’ AgentX our p90 interactivity in Qwen is close to SGLang’s at four and eight concurrent sessions: we’re 1.1x ahead, not 12.8x. The lead opens up at one, 12, and 16 sessions. The full tables are public, so you can judge for yourself. This can still equate to faster AI without spending on new infrastructure.

Agents shouldn’t take long to get started. That’s the difference between getting work done and giving up. In a single session, SGLang's published p90 time to first token with Qwen is 10.6 seconds. Ours? Just 0.78 seconds. You can get near-instant agentic AI responses while saving money on hardware.

Full precision shouldn't be the slow option. FP8 quantization makes a model smaller and faster to run, but at lower precision. We ran a full BF16 Qwen model, with no quantization and no precision loss, and still posted consistently higher numbers than SGLang. You can get the most accurate results without paying for more infrastructure.

Time to first token is the wait before anything happens. For an agent chaining dozens of calls, that wait magnifies. Backboard cuts that delay by an order of magnitude. We reach that first token in 0.78 seconds with one session in Qwen where SGLang takes 10.6. It’s up to 13.6x faster, and ahead at every concurrency level we tested. Quicker answers and agents can translate to lower GPU costs and using only the infrastructure you have. ow.ly/IJSu51059rJ

Interactivity is how fast tokens stream once generation starts. It's the difference between an agent that feels instant and one that stutters. In AgentX, Backboard's p90 interactivity versus SGLang is up to an order of magnitude better: 12.8x in one session, 5.0x at 16 sessions. See our full results in the blog post. ow.ly/zmRx51053g1

Anything > 1.0X = money being left on the table.

Our benchmark rules are simple: Benchmark what you built, not what you built for the benchmark. Hold the model, hardware, workload, and measurement constant. Publish the failures and the trade-offs alongside the wins. Everything we share follows those rules.

More useful work from the same GPUs. Using Qwen3.8-Flash-Next on eight H100 GPUs. Backboard managed up to 2.2× higher output throughput per GPU than the SGLang results on SemiAnalysis AgentX. This was done at BF16 versus SGLang at FP8. Better throughput without sacrificing model precision.

Waiting for the first token shouldn't slow down an AI agent. Backboard offered up to 13.6× faster p90 time to first token than the SGLang results on SemiAnalysis AgentX, running Qwen3.8-Flash-Next on eight H100 GPUs. And Backboard did it at BF16 precision, versus SGLang's FP8.

Agentic AI needs more than fast tokens. It needs responsive tokens. On SemiAnalysis AgentX, Backboard delivered up to 12.8× higher p90 interactivity than SGLang on Qwen3.8-Flash-Next across eight H100 GPUs. That’s quicker starts, smoother answers, and a more consistent experience. Backboard ran BF16. The SGLang reference ran FP8.

More concurrent requests shouldn't mean giving up throughput. On Qwen3.8-27B, Backboard maintained up to a 1.26× output-throughput advantage over the fastest results from vLLM 0.33 and SGLang, with up to 8K inputs and concurrency up to 64 users. All on a single RTX PRO 6000 Blackwell GPU.

Your inference engine should be consistently fast, not just in a few conditions. Backboard won 102 of 104 measured InferenceX metrics against SGLang on Qwen3.8-27B using a lone RTX PRO 6000 Blackwell GPU. That includes throughput, time to first token, and generation speed. This is performance that stands up to scrutiny.

You want your AI inference to win virtually every time. Ours does. Backboard won every measured InferenceX metric against vLLM 0.33 on Qwen3.8-27B, across test configurations covering up to 8K inputs and 64-user concurrency. That’s with one RTX PRO 6000 Blackwell GPU, BF16, and one Backboard configuration.