Prime Intellect (@PrimeIntellect) header
Prime Intellect (@PrimeIntellect) profile picture

Open Superintelligence Factory.

Joined June 2020
Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)
Prime Intellect retweeted
Prime Inference running Whisper faster than Groq. 🤔 I wonder how they did it.
14
8
1
262
10,306
Prime Intellect retweeted
very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong! beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN! ricursive.com/ripple
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
29
15
3
201
12,482
Great points on issues with existing evals, what we did about them, and some nuggets on how difficult it is to prevent reward hacking, especially in the era of multi-agent systems.
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
4
2
36
2,378
Prime Intellect retweeted
We evaluated different leading evals > popular public evals have brittle graders rejecting correct answers, ambiguous tasks + leaked solutions > once fixed many benchmarks are basically saturated > frontier models actively reward hack: reading other models graded rollouts, splitting actions so each step looks benign > all fixes open-sourced
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
20
8
119
7,246
Prime Intellect retweeted
Amazing BenchmarkGate [1] reveal by @xeophon where he found serious (even comical) problems in popular benchmarks: AutomationBench: "[forces the models to] describe their request bodies in prose rather than in JSON Schema" 😅 He also fixed a number issues with the grader, which when combined with the above result in a near saturation of the benchmark! OfficeQA Pro v2: the grader uses a regex and Python-based extraction, which results in rejecting many valid answers. Additionally, there is evidence of models using web access to retrieve the answer key from HuggingFace (receipts are in the post). With these and other fixes, this again results in near saturation of the benchmark! It's very impt to look at data and not trust metrics from third parties. Better yet, you should rely on your own private evals. [1] BenchmarkGate: the gap between the [un]trustworthiness of benchmarks and the extent to which people rely on them.
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
57
16
4
125
10,183
Prime Intellect retweeted
We looked at the data and the data stared right back at us 🥀 Some insights into AutomationBench, OfficeQA Pro v2, web access in public benches and why monitoring is hard. This has been in the work for months, more to come!
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
18
17
3
263
20,618
Prime Intellect retweeted
Evals built by @PrimeIntellect residents, fully open: - MazeBench by @patience_cave - PMPP-Hard by @myainotez - PretrainGym by @HuNtEr_J1324 - OpenTTD-Bench by @thesophiaxu [and even more :) ] Super happy to support such amazing work, check it out!
I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
16
25
1
363
17,789
Prime Intellect retweeted
I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
51
38
13
443
35,691
Prime Intellect retweeted
🚀 0 → 120 BILLION tokens/day in just ONE month! Just one month since onboarding Prime Inference to OpenRouter, we've scaled from zero to 120B tokens served daily. Note this is just a small portion of Prime Inference 🔥 openrouter.ai/provider/prime…
14
19
7
141
24,410
Prime Intellect retweeted
curious people and people interested in llm evals, benchmarks, high data circles, please wake up. my blog post "SWE-bench Multimodal v2 is not multimodal" is up now. sankalp.bearblog.dev/swe_mul…
44
33
15
420
43,496
Prime Intellect retweeted
Introducing AerialSim, a reinforcement learning environment developed with @PrimeIntellect to train and evaluate vision-language agents for autonomous drone control. 🧵
3
8
2
91
6,272
Prime Intellect retweeted
Can AI Agents find every dividing cell in a tumor image? I built Mitosis Bench (dylantirandaz.com/projects.h…) to find out... 🧵
14
9
3
80
4,771
Prime Intellect retweeted
Earlier this year, @a1zhang introduced the idea of compositional generalization, showing how harnesses can help LLMs generalize to harder tasks by decomposing them. In our new blog post, we show that harnesses for Vision Language Models (VLMs) are compositional generalizers too, turning difficult image-based questions into simpler observations the model can solve. (1/8) harishkk.com/writing/visual-…
5
14
2
124
9,419
Prime Intellect retweeted
RLM + continual harness + multi-agent communications is what compose the prime agent harness developed by @PrimeIntellect which now sits at a gazillion stars on github (22.3K). I was kind of already mesmerized by the idea of having an RLM write codes to organize and pick the context to solve problem and to delegate subtask to sub agents; but ON TOP of that to have the harness around the RLM be modifiable by the RLM itself and the sub agents being able to communicate together took it to another level of interest. the ideas of creating a limited set of primitives that fit with the model shape and letting the main agent just figuring out how to run them sound just so right to me. BUT the usual assumption earlier in 2026 was that the harness is temporary thing. Meaning as models get smarter that scaffolding will eventually gets absorbed into the weights. YET we have seen in late 2026 how absolutely essential a solid harness with good primitives are which can be steered by the model itself. that’s why I sat down in september with @sethkarten from princeton and prime intellect to take a deeper look at how all of this stuff works. the team productionized @a1zhang recursive language models and bolted a continual harness around, where the agent edits its own skills, memories, and prompts while it runs, plus persistent sub-agents that message each other directly. the kicker here is that if you read the paper thoroughly you get this feeling in the back of your mind that what you are reading is actually a computer science paper. akin more to something a operating system PhD would put out than anything ml related. we got confirmation from seth himself that this was intentional. the whole thing is designed like an OS and the manipulation at different level like weights, context, REPL, and disk form a memory hierarchy, and each level has its own update rule. in this 2h30 we get into how it works, the results on ARC-AGI-3 (95.5% with opus 5 on the public set, vs ~30% in the official harness), the week-long factorio run with 633 sub-agents, why letting an agent edit its whole system prompt makes it delete itself (hahaaaaa), why models cheat, and a live demo spawning a whole bunch of sub-agents on stream! here is the full run down (we even got a demo): - 00:00:00 : prime agent = RLM + Continual Harness + Agent Communications - 00:03:24 : Seth background and past research! - 00:04:50 : how do you feel about harness field crystallizing into a distinct research discipline? - 00:06:52 : Seth benchmarking journey - 00:09:41 : The issue with games and tendency of reward hacking with frontier models? - 00:12:30 : frontier models and their taste for exploration + LLM play pokemons. - 00:17:30 : would harness capabilities be integrated within the models at the limit? - 00:27:00 : we’re reading a computer science paper actually! - 00:28:40 : prime agent: a self-improving RLM harness - 00:28:56 : a raw llm is just a sequential processor - 00:29:46 : harness adds persistent state, tools and compute - 00:34:15 : issue with zero sum data - 00:36:25 : system state spans weights, context, sessions and disk - 00:42:10 : the system is von neumann like - 00:45:03 : expressive harness lets model build it’s own worfklow - 00:47:03 : each RLM call creates a persistent subagent session - 00:54:57 : does the orchestrator always go idle? - 00:57:50 : what problems are we solving with each of the primitives for long-horizon control mechanisms? - 01:04:35 : are there plans to make public the Prime Intellect traces or all of this will be limited to first party model release? - 01:06:35 : is c-python kernel a better sole tool for models than a traditional JSON tool? - 01:07:55 : refine updates the harness from multiple trajectories - 01:12:18 : direct messages coordinate recursive sessions - 01:15:20 : what are the limit of the continual harness? - 01:20:00 : the figure 2 is suuuuuuuper important guys!!! + continual harness lore - 01:30:15 : three research questions to test the design - 01:32:13 : prime agent scales to 95.5% with more output token on ARC-AGI 3 - 01:34:08 : how did the harness evolve during the ARC-AGI 3 run? - 01:39:05 : ablation on the three pillars of prime agent? - 01:40:40 : harness stuff is getting serious!!!!! - 01:43:45 : more evals for long context and more - 01:49:20 : out-of-loop experiments results - 01:53:40 : PMPP-Hard (GPU Kernels) - 01:54:35 : automated factorio runs! - 01:57:30 : is the monitoring in place for exploit like seen in factorio is enough to avoid them? - 02:05:10 : mazebench result (3D maze that is super duper hard) - 02:07:20 : why the heck are models giving up sooooo early man. - 02:13:15 : there is a whoooole lot to be done still. - 02:14:40 : at which training stage does introducing the harness have the greatest impact? - 02:16:35 : during introduction of harness in training would decomposition of abstractions make sense? - 02:20:20 : prime agent demo 🎉🎉🎉 - 02:29:10 : how do you personally handle the context between multiple hierarchical agent? - 02:31:35 : how do you go about handling multi-agent trajectory for training models? - 02:35:27 : what’s next for seth + prime agent! stuff like this fully build in the open is what make machine learning / ai field such a pleasure to work in man big thanks to the team!
27
24
5
266
12,949
Prime Intellect retweeted
Introducing Inference Router: enterprise-grade inference on demand. Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Qwen3.8 Flash, Muse Spark 1.3, Mercury 3 and more, available with an uptime SLA of at least 99.9%. US hosting and web search built in. you.com/router
32
52
21
463
4,444,607
Prime Intellect retweeted
The next frontier is AI that improves itself, and we believe everyone should know how to build it. This semester, Ao @ao_qu18465 and I are teaching 'How to AI an AI for Almost Anything' @MIT @medialab -- a new course on self-evolving AI systems that recursively improve from experience, feedback, memory, and exploration over long interaction horizons. The key idea: intelligence shouldn't stop improving when training ends. We cover four levels of evolution: (1) context evolution through prompt optimization and context management, (2) harness evolution through memory, tools & skills, workflow optimization, and multi-agent systems, (3) model evolution through RL, distillation, test-time training, self-play, and continual learning, (4) task evolution through generating increasingly complex synthetic tasks, simulations, evaluations, and environments. Slides, readings, and videos available: mit-mi.github.io/how2ai-cour… We thank @Kimi_Moonshot and @PrimeIntellect for generously sponsoring API credits!
45
160
13
1,186
66,950
Prime Intellect retweeted
We’re releasing more open models to help teams across Europe build AI with their own data, on infrastructure they control. Announced at GTC Berlin, the models help developers, researchers and enterprises build specialized AI agents, train robots, design proteins and develop quantum computers. See what’s new: nvda.ws/4iwBMvF
33
51
32
399
31,505
Prime Intellect retweeted
applied research at prime intellect is both a client-facing team and a fundamental research team. continual learning is our north star. we view this dual approach as necessary in order to study the core challenges faced by models which self-improve in production. the real world is big and messy. we spend part of our our time working with customers to train SOTA models for specialized domains, and we automate and productize our work as we go. we also do open-ended research on novel methods for self-improvement and long-horizon memory, and we get to experiment with data that no other lab gets to see. today, the most readily-deployable form of continual learning is converting "exhaust" (agent traces, logs, feedback) into robust environments and tasks for continual RL. but this is not the final form. there are major open challenges around knowledge acquisition, harness-weight tradeoffs, user modeling, dynamic underspecified preferences, updating and planning across timescales, selective forgetting, data efficiency, learning from golden answers, and company-scale simulation. this is the roadmap, roughly speaking. some of these may require changes to architecture, or parameter-efficient methods, or fundamentally new algorithms. we have our hypotheses and early evidence, but everything is on the table. the world is our oyster. we are betting on a radically different future of what ai deployment will look like, and we can see the path more clearly every day. it is a world where agents learn like humans do, with billions of individual models compounding their learnings from the real world continuously. we're hiring incredibly ambitious researchers in SF, NY, and London. come join us.
11
9
2
215
22,201
Prime Intellect retweeted
over the past few months we've been rapidly scaling our partnerships with leading companies across a bunch of incredibly exciting domains we collaborate with their in-house experts to deploy continual learning loops in prod hiring for many new roles :) primeintellect.ai/careers
11
17
2
293
21,556
Prime Intellect retweeted
Excited to announce that I've joined @PrimeIntellect! Our mission is to build the open training stack and let people own their intelligence. Prime Intellect has always gone a step beyond open-weight models and strived for a fully open-source ecosystem by sharing everything you can think of: training infra, data, evaluations, inference, algorithms, harnesses, and more. In just a few weeks, I’ve already learned so much from an incredibly goated team with deep expertise across the entire model stack. Looking forward to building frontier open models with the team. Watch out for what we are cooking, big things ahead!
43
15
452
18,369