A high-signal data ecosystem leveraging data from industry & academia to deliver post-training to frontier labs, neolabs, & enterprises building advanced models
San Francisco, CA
Joined May 2026
- Tweets441
- Following294
- Followers107
- Likes250
ReasonCore AI @ NeurIPS 2026 retweeted
Created a "benchmark benchmark": a frequency distribution of which benchmarks were reported by the frontier labs in 2026: claude.ai/artifact/22cBPJt6A…
Some observations:
- HLE, SWE-Bench Pro, DeepSWE, and Terminal-Bench saw universal adoption
- Scale AI is behind two of those four (SWE-Bench Pro, HLE w/ CAIS), plus MCP-Atlas (#9 benchmark)
- Of the frontier labs, OpenAI is the most prolific benchmark producer (SWE-bench Verified, BrowseComp, GDPval); people don't seem to mind evaluating on a benchmark made by a competitor
- Anthropic has just one on the chart (BioMysteryBench) that basically nobody uses 🤔
- In academia, strong showings from Princeton (SWE-bench, CharXiv), Stanford (Terminal-Bench), Berkeley (CyberGym, Agents' Last Exam), and NYU (GPQA)
- DeepSWE saw incredible adoption, from first appearing in June to all 12 labs reporting it
- Labs cycled through four versions of Terminal-Bench (2.0 → 4.0) in a single year
ReasonCore AI @ NeurIPS 2026 retweeted
Long horizon tasks are getting longer and longer!!!
Any prediction for 2030 and beyond: what will be a long horizon task then?
ReasonCore AI @ NeurIPS 2026 retweeted
Adversarial Speculative Reasoning Improves Long Horizon Agents
arxiv.org/abs/2611.05345
ReasonCore AI @ NeurIPS 2026 retweeted
OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery
arxiv.org/abs/2610.37739
ReasonCore AI @ NeurIPS 2026 retweeted
A team of 70 physicists spent 3 months packing 9000 researcher hours into the FrontierPhysics benchmark
Here's a 3min video made by the official team that explains what's going on. Enjoy!
ReasonCore AI @ NeurIPS 2026 retweeted
You don’t need long context [New paper from @nvidia]
arxiv.org/pdf/2611.05306
ReasonCore AI @ NeurIPS 2026 retweeted
Introducing FrontierPhysics - a benchmark evaluating agents for end-to-end physics research
Our core team from physics labs at Berkeley, Caltech, and Harvard recruited 42 physicists from 26 institutions. 28 of them published papers on Nature, Nature Physics, PRX, PRA*
1/7 🧵
ReasonCore AI @ NeurIPS 2026 retweeted
Good to see @ArtificialAnlys, @MiniHui_zhu and @MinyangTian1 bringing CritPt grading to more research teams. A shared grader makes it easier to test an idea, measure the gain, and build on what works.
That's a useful step for research-level physics training. ReasonCore has 5,000 CritPt tasks available for teams working on that next jump.
We’ve worked with the CritPt authors to finalize the CritPt v2 grading server and will be making this available to the community via Artificial Analysis’ API tomorrow. As part of this process the authors made robustness improvements to the grading server logic, which we have incorporated as v5.0.1 of the Artificial Analysis Intelligence Index
We regraded every model and observe small score increases for most models. Intelligence Index rankings are unchanged among frontier models, with scores moving by at most 1 point.
Thanks to @MiniHui_zhu, @MinyangTian1 and the rest of the CritPt team for their ongoing collaboration
ReasonCore AI @ NeurIPS 2026 retweeted
Babysitting AI Agents:
A User-Centered Taxonomy and Study of Agentic Delegation Failures
arxiv.org/html/2610.34748v1
ReasonCore AI @ NeurIPS 2026 retweeted
AgentBaton: Efficient Model Collaboration in Agentic Tasks
arxiv.org/html/2610.35959v1
ReasonCore AI @ NeurIPS 2026 retweeted
I looked at who actually follows me here. About 1100+ followers, public bios only, aggregate counts.
Half hold or are pursuing a PhD. 57% are students or academics. 15% have worked or interned at a frontier lab. 39 followers work on evals, training data, and RL environments.
Thank you for being part of it 🙏.
Which panel are you in?
If you like what I post, let me know.
ReasonCore AI @ NeurIPS 2026 retweeted
@yus167 - Great work!
I thought of sharing this visual (made with AI) to understand and summarize.
ReasonCore AI @ NeurIPS 2026 retweeted
Artificial Analysis Intelligence Index v5.0 came out two days ago. Not only did Terminal-Bench-Science make its debut in the index (15% of the total index), but the entire “Science” category was also up-weighted from 20% to 25%.
Scientific research is more important than ever, but researchers aren’t always using the most intelligent models to power their research agents. Some interesting trends:
- Anthropic and OpenAI models still dominate the bulk of usage, making up the 6 of the 7 most popular models
- Deepseek V4.1 Flash is the most popular open and Chinese model by a wide margin. Its relatively cheap intelligence is quite popular among researchers.
- GPT-6 Luna’s usage punches furthest above its intelligence score while Haiku 5.5 is most underused for its intelligence
- Despite recent splashes, Google has yet to make meaningful inroads with its latest Gemini 4 models. It would probably help to make the models usable on launch day instead of making researchers wait 2 weeks.
ReasonCore AI @ NeurIPS 2026 retweeted
Language models can learn from the open internet. Where does Physical AI get its ground truth?
Unlike digital AI, autonomous machines need data grounded in real-world behavior, 3D environments, and complex decision-making. That kind of data is much harder to produce at scale.
This blog, “Closing the Ground-Truth Gap in Physical AI”, explores why this ground-truth gap matters and how Data Factory, part of our HyperFoundry™ Physical AI development platform, turns real-world driving into machine-generated ground truth, from 3D scene labels and reasoning traces to reconstructed environments.
Read more: plus.ai/news-and-insights/20….
ReasonCore AI @ NeurIPS 2026 retweeted
Here's an example of failures, jerks in capturing that may be good or bad to the training data, which egocentric data cannot get
ReasonCore AI @ NeurIPS 2026 retweeted
Most robot demo datasets don’t fail loudly. They fail quietly, as a policy that “almost works.” Here are the checks we run before any episode goes into training 🧵
Sync. Camera, joint and gripper streams within 5 ms. A 20 ms skew looks like the robot closing before it arrives, and the policy learns exactly that.
Frame drops. Count them per episode. A handful of dropped frames during a grasp is worse than many during transit; we reject on where, not just how many.
Gripper consistency. Different operators close differently. Normalize, or your policy learns to hesitate.
Joint limits + resets. Episodes that touch limits or start from a “lucky” pose inflate success in eval and fail on the real table.
Failure → recovery pairs. Keep them, label them. They teach the policy what to do after the first mistake, which is where real deployments live.
None of this is glamorous. All of it shows up in success rate. What’s the check you added after getting burned?
#PhysicalAI @LeRobotHF #teleoperation
ReasonCore AI @ NeurIPS 2026 retweeted
Most robot datasets don’t fail from too few demos. They fail on boring stuff:
-->camera and joint streams drifting >5 ms apart
-->dropped frames that silently stretch an action
-->gripper state that disagrees with the video
resets leaking into the next episode
@IntuistAI checked @DecisionFacts 200-episode SO-101 teleop set before training:
189 keep · 10 trim · 1 drop
5 episodes skipped frames while timestamps stayed even.
#Robotics #LeRobot #PhysicalAI