DataMining

AI data stopped being sold as answers this year. We build code and agent tasks that frontier models still fail, and publish the traces. Six notes on what changed on the buyer's side — and what we measured.

Nav agents get clumsy when memory is explicit: text maps and raw frames lose spatial detail and blow the buffer. JanusVLN uses dual implicit memory, like two hemispheres. One stream asks what; the other asks where. It does not store raw, high-dimensional observations. After the network processes the scene, only the gist and the spatial feel are kept — closer to how you walk without replaying every frame. If your agent memory is a dump of traces that will not replay, that is an environment problem, not a prompt problem. We sell hard agent environments — task, context, action space, grader — with a reference that passes. Not answer dumps. DM. #JanusVLN #AgentMemory #EmbodiedAI #VLN #AIResearch

If I only keep three agent surveys, these are the ones — biased toward agentic RL. 1) Foundation Agents — brain-inspired → evolutionary, collaborative, safe systems 2) Agentic RL for LLMs — the landscape survey 3) Self-Evolving Agents — on the path to ASI Titles (as listed): • Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems • The Landscape of Agentic Reinforcement Learning for LLMs: A Survey • A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence A map of the field ≠ a locked task, context, action space, and grader. Surveys tell you where the literature sits. Hard agent environments tell you whether a stack actually finishes under a locked harness. We sell the latter — with a reference that passes. Not answer dumps. DM. #Agent #AgenticRL #LLM #AIResearch #OpenSource

Buying answers is over. Labs now pay for the process: trajectories, replayable environments, and rubrics that survive a model actively trying to game them. Through mid-2025, data was mostly "write a clean solution, ship it." By Q4 the easy sets had no marginal value left. Demand moved to long-horizon systems work, CUA, PCB layout, frontier math, Terminal-Bench-class suites. A model cannot learn a job from its final answer. Nobody fixes a multi-host outage or routes a 12-layer board on the first attempt — experts spend the day reading errors and backing out of dead ends. That detour log is the signal, and it is not on the public internet. So vendors buy hours from people who can do the work: $200–300/hr for a math PhD or a senior PCB engineer, with capture tooling recording cursor, keystrokes, deleted code, hesitation. Money is not the bottleneck — senior people cannot articulate their own shortcuts. Rebuilding the environment so the action replays is the next wall: WebHarbor cloned 153 real sites with login state; PCB work needs industrial simulators inside the container. Then rubrics, and it gets ugly. Acceptance is one line: did the model get better. - Vals AI caught a trending open model reading answers in two-thirds of its long-horizon coding tasks — the env still had git history. - Scale AI cut 89 tasks from SWE-Bench Pro V2 after audit. - Handshake: models spend >80% of reasoning on "what will the grader test." - Static rubrics decay — write long enough and you collect points. Hence Anthropic's GRAFT. So vendors now run half a training loop before delivery: stand up the env, run open models hundreds of rounds, hunt fake trajectories, deepen the rubric. Co-evolution. Without a training researcher in-house, you cannot explain why your data bounced. Embodied is harder. Real-robot factories do not transfer across hardware; handheld capture is cheap — AIRoA-0 logged 16,151 hours and still transferred across robots with zero real-robot data — but five-finger mapping loses fidelity. World models flatter outcomes. Acceptance stays contested. Mercor mapped 923 occupations: ~90% of US work time is uncovered by existing benchmarks. Labs will not chase it; enterprises will, via post-training. The vendors who survived lab-grade acceptance build that pipeline. We build the unglamorous middle: hard trajectories, replayable environments, anti-gaming rubrics. No answer dumps. DM if your eval keeps scoring high and shipping nothing. #DataMining #Trajectory #Evals #CUA #EmbodiedAI

Mercor put Gemini 4 Carbon #1 on APEX-Agents: 84.2% Pass@1, 90.4% mean. A buyer asked: where is pass^8? At launch a leader ran about half on it. Pass@1 is a poster; reliability under retries is the product. We sell tasks frontier still fails, reference passing a locked grader.

Mistral Large 4 weights claim #1 open on FrontierCode. AA has it at 15% on FrontierCode 2.0, behind other open runs; Coding Agent Index 48 on half the tokens of the open leader. Same model, different harness + grader. We sell tasks frontier still fails under a locked harness.

FrontierCode 2.0 asks a sharper question: not "does it meet the spec?" but "would you merge the PR?" Meeting a spec is one job. Beating it without breaking it is another. We sell that cut: design not in the prompt, frontier still fails, reference passes under a locked grader.

GPT-6.1 Terra moved the cost frontier. Coding Agent Index still sits 6 points under Sol. Cheaper tokens are not the hard tail. We sell tasks frontier still fails under a locked harness, with a reference that passes.

SWE-Bench Pro V2 hits 99%. Audit: prompt names ~81% of the new functions hidden tests call. Issue text names ~10%. So it grades spec-following. Writing the spec was the hard part. We sell that cut: design not in prompt, frontier still fails, reference passes a locked grader.

Vals put Kimi K3.1 #1 among open-weight models. The gain is narrow — Terminal-Bench and Code Migration carry it, the rest barely moves. We care about the hard tail that still fails under a locked harness, with a reference that passes.

Coding Agent Index v2.0 got harder: FrontierCode v1.1 replaced DeepSWE, and stricter QnA grading cut configs up to 7 points. That's the story — not another Verified screenshot. We sell the hard tail: tasks frontier mostly fails, where a reference passes under a locked grader.

74% on SWE-bench Verified and ~55% on Terminal-Bench is not a model story. It is a harness + horizon + grader story. We sell the hard tail: locked action space, reference PASS, and tasks where frontier still fails most attempts.