co-founder @daridotdev // prev @ycombinator, @caltech, basketball data @lakers, software @uber, startups
Fremont, CA
Joined April 2018
- Tweets125
- Following701
- Followers726
- Likes4.4K
Pinned Tweet
Today, we're releasing our open-weight, auto-routing model @daridotdev, built for coding agents.
We're state-of-the-art on the Pareto Frontier, w/ 70% cost reduction + comparable coding performance to Fable.
Bring your own evals, choose your models, or use our defaults.
They spent millions on router dot com.
We spent $20 on routerer.com.
We saved money on the domain to save you money on tokens.
> Pool Claude + Codex subs
> Pick the right model + thinking level w/ our SOTA router
> Use your traces to fix token waste
> Open-source
For small teams, subs are the best way to save on tokens.
Route based on how much usage is left, cache warmth + automatically claim resets.
Run subscription pooling fully locally, or use free hosted to get detailed analytics across your team.
github.com/mupt-ai/dari
1
164
Are OpenAI models not the same on sub / API?
Ran GPT-6 Luna (high) on DeepSWE 1.1 in Codex:
> With subscription: 26% solved
> With API (via Vercel AI Gateway): 41% solved
Thought it might be variance, but the sign-in run also used ~4x fewer reasoning tokens per task
DeepSeek-V4.1-Flash-1017 is Pareto-mogging in our initial tests.
It's the first open-source model we've tested that's consistently on the Pareto frontier in real-world PRs, running in @pidotdev
Some results from Sentry, PostHog, and Next.js PRs:
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
View tasks and traces here:
- selfbench.dev/vercel/next.js
- selfbench.dev/PostHog/postho…
- selfbench.dev/getsentry/sent…
Haiku 5.5 is an incredibly capable model at its size.
We ran Haiku on real Pi, Next.js, PostHog, and Sentry tasks, and it was on the Pareto frontier in every test.
Slightly higher $/task than Luna for increased performance.
Towards intelligence too cheap to meter!
Open-weight models may be costing you more on real work.
We built self-bench to turn real GitHub PRs into evals.
Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable.
Find the best model for your repo: selfbench.dev
Is @MistralAI Large 4 smart or just chonk-y?
We ran Le Chonk on real PRs from Next.js, Sentry, Pi, and PostHog.
> Mistral Large 4 generally benchmarked worse + similar price as Kimi K3 and GLM 5.3 in the Pi harness.
> None of the 3 models were ever on the Pareto frontier.
Open-weight models may be costing you more on real work.
We built self-bench to turn real GitHub PRs into evals.
Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable.
Find the best model for your repo: selfbench.dev
On our eval of 82 tasks from real @sentry PRs, GLM 5.3 was
> 50% more expensive than claude opus 5.5 ($1.83 vs $1.22 per task)
> 8.5 points less accurate (74.4% vs 82.9%)
Open-weight models may be costing you more on real work.
We built self-bench to turn real GitHub PRs into evals.
Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable.
Find the best model for your repo: selfbench.dev
Open-weight models may be costing you more on real work.
We built self-bench to turn real GitHub PRs into evals.
Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable.
Find the best model for your repo: selfbench.dev
it’s all goodhart’s law (when a measure becomes a target it ceases to be a good measure)
we’re killing trust in every benchmark like russell westbrook killed trust in the triple double
if you're running out of Fable weekly limits try running
`dari --claude`
Use Fable + open models and make your subscriptions last for 2x longer with no quality degredation :)
Literally unusable. The rate limits are absurd. Oh, and by the way, Fable’s automatic continuation is bugged and doesn’t even work.
I honestly don’t know why I still bother using Claude at this point. 5.6 is simply better overall anyway.
Give me GPT-Astra and im fine. its so frustrating. seriously.
oh, and btw. For subscription users, Anthropic has not announced lower prices or higher usage limits regarding Fable 5.1s efficency gains; the savings explicitly apply “wherever usage is billed by token,” so greater efficiency within Pro or Max subscriptions possible not gonna happen.
some thoughts re: subagents
subagents are *great* if you’re selling tokens. increasing token throughput is a massive win for a token seller.
nobody has the bandwidth to manage 100 agents concurrently, but if all it takes is one prompt and “ultracode”, you can easily 10x usage!
hot take: you dont need subagents for most work (esp wrt coding agents)
yes, it can save context, compaction is fine
yes, it can make fanout better, except coordinated work often still sucks
yes, its good for adversarial verification, but you prob are doing that elsewhere
1
8
883
Model routers and evals go hand-in-hand - router decisions should be based on evals, and you need to eval different models/harnesses within the exact (or very similar) environments that your agents act in to properly measure savings
this is why we built self-bench! (github.com/mupt-ai/self-benc…) - it's fully open-source and lets you build simple @harborframework evals from your PRs
2
7
271
Public evals are saturated. What matters is that a model reliably works in your codebase.
To solve this, we built self-bench (github.com/mupt-ai/self-benc…) - an open source package to automatically build evals on your own private PRs.
One CLI command -> get a clean Harbor dataset.
Companies are trying to reduce token costs by using smaller / open-source models, but switching without evals is flying blind.
We built evals based on some popular OSS packages like Next.js and Vite. @daridotdev 's auto-router is on the pareto-frontier in all of our tests!
1
4
385
in response to funding cuts, universities should just require every phd student to spend n months generating rl environments and training data, then sell it in bulk to frontier labs
a modern day robinhood story
1
10
852
another reason to feel like NBA teams / sports orgs are undervalued:
teams capture every player's 3d skeletal data (hawkeye data) at ~25hz in games / practice. if players wear eye-tracking goggles (there are real coaching incentives to this) you get real-world egocentric data
Training world models needs egocentric video and dense action signals, synchronized. That data is genuinely hard to find.
We built it from Counter-Strike 2 demos.
CS2-10k: 600K+ player-round videos, 10K+ hours, per-frame annotations — keyboard state, mouse delta, 3D position, camera yaw/pitch. All paired to the visual stream.
Why CS2 demos? Matches are deterministic replays. We can reconstruct first-person video and extract the exact control inputs that caused every visual change. No labeling, no estimation.
We're also releasing cs2-dem-renderer — the open-source pipeline we used to build it. Give it a .dem file, it outputs .mp4 + .parquet. Run your own dataset at whatever scale you need.
1
7
696
Token-maxxing is getting out of hand.
Most AI apps send every request to the biggest model, even when a smaller model would work.
We built Dari Router to fix that.
@daridotdev gives you full control over your routing workflows.
Use your own traces and evals to build custom routers for each workflow: fine-tuned small-model routers when you need learned behavior, or deterministic policies when you want full control.
1
10
853
there are several interesting basketball benchmarks one could do but would be cool to see how well LLMs perform at predicting rotation charts
imperfect information, but given recent stats, lineup data, etc, how close can an LLM emulate coach game-planning
We need more niche benches.
We need ios-bench.
We need ts-bench.
We need baseball-bench.
We need yt-thumbnail-bench.
We need way more creativity in how we measure what models can do.
2
11
980
Everyone seems to only focus on sandboxes for agent runtimes
but decoupling your agent loop from the sandbox / persisting state via durable execution is tablestakes for long-running agents, as seen w/ Cursor
if you want to scale long-running agents try us out @daridotdev!
A great cloud agent experience involves a lot more than moving a local agent to a server.
We've learned that it requires a durable execution platform, a powerful harness, and the tools and infra to give agents realistic development environments.
cursor.com/blog/cloud-agent-…
7
515
Built a tool for sharing images/files to my remote agents: contextdrop.dev
Upload a file and get a temp link or login on both machines and let agents pull files.
I liked the simplicity of @bentlegen’s glance.sh but wanted something similar as a pure CLI.
6
268