Organizing human intelligence to power the AI economy.
San Francisco
Joined April 2021
- Tweets509
- Following31
- Followers30.4K
- Likes916
Pinned Tweet
Mercor has raised a $750 million Series D at a $20 billion valuation to scale our mission of organizing human intelligence for the entire economy.
Every company now needs what we built for the frontier AI labs. Over the next five years, more money will be spent applying AI to real work than building new models, and every company will need evals and post-training built on how its own people work.
Thank you to our 5+ million experts and the customers who have built Mercor with us, and to our investors, including @benchmark, @felicis, @fundomo, funds managed by affiliates of Apollo, G Squared, @generalcatalyst, @GladeBrookCap, @nvidia, @salesforce, @SapphireVC, @StepStoneVC, XTX Ventures, and @Zoom.
Thank you to our team of time travelers who turn our ambitions into reality. And to those who want to build with us, we’re hiring.
Recently, we released TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work.
Following this study, we surveyed professionals in the same six occupations asking which tasks they would hand to AI, which they would do themselves, and how much AI has changed work for them.
We found that across occupations, workers prefer to keep human-centered tasks like teaching emotional skills and interviewing clients, and hand off clerical work like documenting patient procedures.
Read the blog post: mercor.com/blog/what-workers…
Muse Ember scores 64.3% Pass@1 (#16) and 75.7% mean score on APEX-Agents.
Muse Ember is Meta’s smallest multimodal model, built for high-volume agent work: tool calls, coding subtasks, routing and document extraction. On our long-horizon professional tasks, it lands within the margin of Muse Wonder 1.0 (66.1%), Meta's top model on the board.
Here’s how Muse Ember compares with other fast, low-cost models:
DeepSeek-V4.1-Flash-1017: 69.8%
Gemini 3.7 Flash: 67.8%
Muse Ember: 64.3%
Gemini 3.8 Flash: 64.3%
Haiku 5.5: 56.1%
GPT-6 Luna: 44.3%
Ember is level with Gemini 3.8 Flash. It is 5.5 points behind DeepSeek-V4.1-Flash-1017 and 3.5 behind Gemini 3.7 Flash, both inside the confidence intervals. It is 8.2 points above Haiku 5.5 and 20.0 above GPT-6 Luna.
By APEX-Agents domain (Pass@1 / mean score):
Corporate law: 68.1% (#10) / 86.5% (#5)
Investment banking: 62.8% (#15) / 67.6% (#15)
Management consulting: 61.9% (#17) / 73.2% (#18)
Law is its strongest domain: the 86.5% mean is the highest for any Meta model on this domain.
Meta on APEX-Agents since July (Pass@1):
Muse Spark 1.1: 31.8%
Muse Spark 1.2: 45.4%
Muse Spark 1.3: 58.6%
Muse Wonder 1.0: 66.1%
Muse Ember: 64.3%
Congrats @AIatMeta.
See full leaderboard: mercor.com/apex/apex-agents-…
Fable 5.2 from @AnthropicAI debuts at #3 on APEX-Agents.
80.1% Pass@1 (#3)
87.6% mean score (#2)
Only Gemini 4 Carbon (84.2%) and Gemini 4 Argon (82.4%) rank higher on Pass@1.
For mean score, Fable 5.2 ranks second at 87.6%, 0.2 points behind Argon (87.4%) and within the margin of error.
Fable 5.2 is now the top Anthropic model on APEX-Agents, ahead of Sonnet 5.5 at 75.5% Pass@1. It scores +11.5 points Pass@1 over Fable 5.1 (68.6%) and +16.5 over Fable 5 (63.6%).
Pass@1 by APEX-Agents domain:
Management consulting: 91.2% (#2)
Investment banking: 80.9% (#3)
Corporate law: 68.1% (#9)
For mean score, Fable 5.2 ties Gemini 4 Carbon for #1 with 94.4% in management consulting. Fable 5.2 also performed well on accounting tasks.
22.5% Pass@1 (tied with Gemini 4 Carbon for #2; Argon leads at 23.8%)
70.4% mean score (#3)
Congrats to the @claudeai team.
See full leaderboard: mercor.com/apex/apex-agents-…
Proud to have contributed to this work with @ElorianAI.
Separating what models fail to see from what they fail to reason about makes benchmarks more useful.
Thank you to @AndrewDai and everyone who helped bring this research to life.
Tired of seeing benchmarks saying we are at AGI but models still can't understand a 3D protein view or a floor plan? When a visual model gets something wrong, is it failing to see or failing to reason about what it sees?
We've built two benchmarks that answers these questions. Explore DecoyCount and ScienceVision here: github.com/elorian-ai/decoyc… and huggingface.co/datasets/Unip…
And thank you @mercor and @UniPat_AI for your partnership in building these!
We helped @thinkymachines evaluate Inkling 2 Small ahead of today's launch. Here's how it ranks on the APEX leaderboards.
APEX-Agents:
46.8% Pass@1
60.9% mean score
By domain (Pass@1):
Law: 62.2% (+12.5)
Management Consulting: 40.3% (+14.8)
Investment Banking: 37.8% (+11.5)
Inkling 2 Small gains +13 points on APEX-Agents overall compared with Inkling (33.8% Pass@1) launched in July.
APEX-Accounting:
14.5% Pass@1
62.0% mean score
It's the second highest scoring open model on APEX-Accounting, behind Kimi K3.1 Pro (62.4% mean score).
Congrats to the Thinking Machines team.
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.
Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.
thinkingmachines.ai/news/ink…
We’ve spent years developing @DoorDash’s delivery network: understanding how local businesses operate, how orders get fulfilled, and how to complete deliveries across neighborhoods with different physical constraints.
Now we’re building proactive agents that can use that infrastructure to execute in the physical world.
@andyfang with @BrendanFoody on The Frontier ⬇️
3
20
3,771
Hy4 from @TencentHunyuan is the new #1 open-weights model on APEX-Agents.
77.6% Pass@1 (#3 of 69)
85.1% mean score (#3 of 69)
Here’s the updated APEX-Agents open-weights top 5 (Pass@1):
1. Hy4: 77.6%
2. DeepSeek-V4.1-Flash-1017: 69.8%
3. Kimi K3.1 Pro: 65.5%
4. Qwen3.8-Max: 63.3%
5. MiMo V2.6 Flash RL: 57.4%
Hy 4 is also the top open-weights model in every APEX-Agents domain:
Management Consultant: 85.6%
Investment Banking Analyst: 75.3%
Corporate Lawyer: 71.9%
Congrats to @TencentHunyuan.
See the full leaderboard: mercor.com/apex/
AI models are rapidly improving on benchmark tasks, and that progress is often read as an answer to a bigger question: can AI do your job?
But benchmarks don’t measure jobs; they measure tasks. Most economically valuable work takes place within companies, where people must coordinate, decide and negotiate to get things done. Jobs are messy in a way that tasks are not.
In a new paper, Mercor Research's @aden_barton and @victorbarres_, along with @NicolaiFoss and Henrik M. Nielsen, develop a vocabulary for this messiness, which they call organizational difficulty, and apply it to ~500 prominent AI benchmarks.
Read the blog post: mercor.com/blog/playing-well…
Read the paper: arxiv.org/abs/2611.03494
The results are striking. Fewer than a quarter of these benchmarks even claim to represent any organizational elements, and only 16% separately report how well models perform on one.
And when they do appear, it’s in very narrow forms. Authority, for example, is almost always a written policy in the prompt. In real work, several overlapping chains of authority may apply, some explicit and some implicit.
The market for AI training data could grow 100x over the next five years.
Frontier labs need increasingly realistic data from actual businesses to train more capable models.
@BrendanFoody explains why this creates a new revenue opportunity for founders and companies.
Find out more at the link in the thread.
Learn more & talk to our team: mercor.com/data/?utm_source=…
Kimi K3.1 Pro from @Kimi_Moonshot debuts as the #2 open-weights model on APEX-Agents.
Top 5 open-weights models on APEX-Agents (Pass@1):
1. DeepSeek-V4.1-Flash-1017: 69.8%
2. Kimi K3.1 Pro: 65.5%
3. Qwen 3.8-Max: 63.3%
4. MiMo V2.6 Flash RL: 57.4%
5. GLM-5.3: 56.6%
The Kimi model family has gained nearly 30 points on APEX-Agents in five months.
Kimi K2.7 Code (June): 37.6%
Kimi K3 (July): 47.8%
Kimi K3.1 Pro (October): 65.5%
Congrats to the @Kimi_Moonshot team.
See full leaderboard: mercor.com/apex/apex-agents-…
At Fireworks Forge today, Mercor's CEO @brendanfoody spoke on how Mercor turns open models into frontier specialists.
The core idea is that training data has changed. In 2020 it meant crowdsourcing: SFT, RLHF and behavior cloning, done at scale by people with no particular expertise. Today, the data that moves frontier models comes from experts. They write the rubrics, unit tests and RL environments that teach a model to do real professional work.
The same expert judgment also measures the results. Our APEX benchmarks test frontier models on real knowledge work. Professionals design the tasks and grade them against rubrics, so we can see where a model holds up on the job and where it falls short.
Experts train the model, and experts check it. That's how an open model becomes a specialist.
See the latest results at mercor.com/apex.
Thank you to @FireworksAI_HQ for having us.
We're proud to have collaborated on a new benchmark built by @GoogleAI: Physical-Engineering's Last Exam (PLE).
Physical engineers design the skyscrapers, power grids, and chip fabs we rely on, yet no existing benchmark tested whether AI agents could do that work end to end.
Together with Google and practicing engineers, we built 85 long-horizon tasks across mechanical, electrical, civil, and structural engineering. On the private test set, all frontier models score below 50%:
1. Claude Opus 5.5: 48.3%
2. GPT 6 Astra: 45.8%
3. Claude Sonnet 5.5: 43.5%
4. Gemini 4 Argon: 40.6%
5. Claude Opus 5: 40.0%
On the tasks we co-authored, the top agent satisfied 91% of individual constraints, but only 41.3% of its deliverable items passed every hard safety check.
Learning how AI fails at physical engineering can help us improve future models and build a safer world.
This benchmark is also an open competition. Bring your own agent setup and submit it for evaluation on our private test set at ple-bench.com
AI models are getting smarter, but there's still a gap between a strong benchmark score and reliable real-world work.
Our Head of Product @OsvaldNitski sat down with @OpenRouter co-founder @cclark at New Defaults in SF to talk about how models learn to do real work, and how expert data and evals get them there.
When your inference bill starts climbing, choosing the right model becomes a question of accuracy, cost, and latency.
In our latest New Defaults, OpenRouter’s @cclark sat down with @OsvaldNitski from @mercor to discuss how evals help teams decide when to optimize, switch models, or train their own.
Gemini 4 Carbon debuts at #1 on APEX-Agents, and it's the only model on the leaderboard with a mean above 90%.
🥇Gemini 4 Carbon - 84.2% Pass@1, 90.4% mean
🥈Gemini 4 Argon - 82.4% Pass@1, 87.4% mean
🥉Claude Sonnet 5.5 - 75.5% Pass@1, 83.7% mean
Google calls Carbon state-of-the-art on long-horizon work. APEX-Agents tests this with multi-step tasks from investment banking, management consulting and corporate law.
Management consulting: 91.9% Pass@1 (#1)
Investment banking: 86.9% Pass@1 (#1)
Corporate law: 73.8% Pass@1 (#2)
Carbon leads in consulting and banking. It trails Argon by 2.9 points in corporate law, where Argon still leads at 76.7% (#1).
Congrats @GoogleDeepMind on the launch.
Congratulations @judgmentlabs on launching general access and thanks again for the @mercor ice cream. 🍨
Mercor developers advance the AI frontier every day. We're proud to work with @cognition as we build the evals, infrastructure, and agents that move the AI industry forward.
Founded by three 22-year-olds in 2023, @mercor crossed $2B ARR in June, only 4 months after hitting $1B.
Today, millions of experts on the platform bring their knowledge of medicine, law, engineering, and beyond to frontier AI - helping models take on increasingly complex and consequential work in the real world.
Mercor's engineers move at extraordinary speed.
They give Devin long-running tasks before bed, during meetings, or on their way to the gym. They come back to tested features and PRs ready to merge.
Join us tomorrow in San Francisco for a free scoop of @Mercor mocha ice cream.🍦
Congratulations @JudgmentLabs on releasing general access and thanks for the sweet treat.
500 Howard St.
San Francisco
Wednesday, October 28, 11AM-1:15PM
Workers in six occupations do 79 tasks the government's official occupational taxonomy has yet to add. That work is worth about $140B a year in wages.
That taxonomy is called O*NET, the U.S. Department of Labor's database of the tasks each occupation involves. Workers use it to help with job transitions, and economists use it to estimate which jobs AI will change. If the lists are incomplete, so are the estimates.
Today we're releasing TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work.
Built with researchers from Stanford, the O*NET Resource Center, and NBER. Thank you to @robinsonaisaac @erikbryn @diyi_yang @jennjwang @andreas_h0wpt @Digeconlab @stanfordnlp @stanfordhai @stanfordAIlab for contributing to this work.
Read the blog: mercor.com/blog/introducing-…
Existing occupational taxonomies often leave important tasks out.
An ER doctor main job may be seeing, treating, and discharging patients. But when an AI interviewer asks what happens next, another task comes up: calling patients back with their results. O*NET doesn’t list this, despite 88% of ER Physicians we surveyed saying they do this.
AI interviewers make it easier to surface these additional responsibilities that constitute a full profession.
TaskNet is designed to work in three steps:
1. An AI interviewer asks workers about their job
2. It asks about tasks they may have missed
3. Workers confirm the final list
GPT-6.1 Terra is OpenAI’s #1 model on APEX-Agents.
68.3% Pass@1 (#6)
79.3% mean score (#6)
Compared to last generation GPT-5.6 Terra, the new model gains 10.1 points on Pass@1.
Management consulting: 80.9% Pass@1 (#2)
Investment banking: 66.2% Pass@1 (#7)
Corporate law: 57.7% Pass@1 (#39)
The model is strongest on management consulting tasks, ranking second overall, behind Gemini 4 Argon (88.4%).
Against GPT-5.6 Terra, it gains 24.7 points in consulting and 10.9 in banking. Law goes the other way, down 5.4 points from GPT-5.6 Terra.
GPT-6.1 Terra still averages 80.6% of rubric criteria on law tasks, but fully completes only 57.7% of them. It gets most of the way and misses the last requirements.
Across 4 runs per task, GPT-6.1 Terra passed 134 of 240 tasks every time and 187 at least once. 22 tasks it never scored on, 14 of them in investment banking.
GPT-6.1 Terra’s strength comes from reading more than its siblings: 735k input tokens per attempt, vs 497k for GPT-6 Astra and 489k for GPT-6.1 Sol. Despite reading more, it costs less.
GPT-6.1 Terra: $0.19 per task ($1/$5 per 1M tokens)
GPT-6.1 Sol: $0.26
GPT-6 Astra: $1.30
It’s also a fast model. The median attempt finishes 1.2x faster than GPT-6 Astra and on par with GPT-6.1 Sol.
Congrats @OpenAI on the launch.
See full leaderboard: mercor.com/apex/apex-agents-…
