MegaRouter

🥇 @AnthropicAI's Fable 5.2 takes the top spot on our Remote Labor Index with @CAIS, our leaderboard which compares AI agents' work head-to-head with work by human professionals on real-world remote tasks. The model successfully automates 41.3% of projects, nearly doubling the performance of the next-best model Opus 5.5, a major leap in AI's ability to handle real-world work. A closer look at where the biggest jumps are coming from 🧵

Sam Altman (OpenAI CEO): "You don't need to write prompts anymore." In just 26 minutes, he explains how to unlock ChatGPT's capabilities in ways most users never explore. During a talk with Stanford students, he shares a different way to think about using AI. After watching it, I realized I was only scratching the surface of what ChatGPT can actually do. Watch the full video to discover how to build a system that can prompt itself.

internal structure of AI companies..

Introducing the Artificial Analysis Decision Index v2: a new proprietary evaluation, updated benchmarks, and a headline metric that rewards model calibration We’ve released the second version of the Artificial Analysis Decision Index to better reflect real-world utility and guard against benchmark gaming. 45% of the Index is now built on private datasets, up from 25% in v1. Drex Max v1.1 from @NaceAI narrowly leads among decision models at 73, within 1 point of GLiDE v1.2 (@fastinoAI) and Mammoth 27B (@baselabs) Updates in Decision Index v2: ➤ Decision Score: We adopt a new headline metric that rewards a model for both correctness and calibrated confidence. Each question scores from 0 to 100: right answers score above 50, wrong answers below 50, and confidence in its top answer moves the score within each half ➤ AA-Paperclip (30%): We added a new private benchmark of 1,500 documents including resumes, receipts, invoices, and AI and machine learning research papers. Models make decisions and extract facts from each document and are scored jointly as a single decision, correct only if all 15 answers are right ➤ AA-Symmetry (15%): This evaluates whether models change their answers for equivalent questions worded differently. We test 120 questions with variations such as reordered options, swapped labels, and negated yes/no questions, then measure how consistent the model’s probabilities are across those variations ➤ AA-Omniscience-Decision v1.1 and HLE-MultiChoice v1.1 (15% each, down from 25%): AA-Omniscience-Decision now varies the wording of its “I don’t know” option across ten alternatives. HLE-MultiChoice adds an option to decline using the same ten phrasings We’ve also added EvalSafe-ONET-AA (15%), our implementation of @typesafeai’s ONET benchmark. We have retained When2Call from @NVIDIAAI in the Index at a decreased weighting of 10% (down from 25%), and removed WildGuardTest given its high saturation Key results: ➤ Three decision models sit within 1 point at the top of the Index: Drex Max v1.1 from Nace AI and GLiDE v1.2 from Fastino Labs both score 73, with Mammoth 27B from Base Labs scoring 72 ➤ Cost per 1M Tasks varies by more than 60x across decision models: Celeris-1.1 Decision ($14), Decider v1.2 ($68) and Drex Max v1.1 ($102) sit on the Decision Index vs. Cost Pareto frontier, while Clef costs $859. Token price is not the only cost driver: most decision models, including Drex Max v1.1 and Decider v1.2, bill a document’s input once for all its questions, while GLiDE v1.2 and Decider Flash v1.2 bill it for every question. This means GLiDE v1.2 costs $780, nearly 8x higher than Drex Max v1.1 despite only 33% higher list prices ➤ Language models can score higher, alongside higher Cost and Time per Task: Claude Haiku 5.5 (Max) scores 77 and GPT-6 Luna (Max) scores 76 on Decision Index v2, putting them ~3 points ahead of Drex Max v1.1. GPT-6 Luna (Max) costs $1,530 per 1M Tasks vs. $102 for Drex Max v1.1 (15x the cost) and takes 11.34 seconds per task vs. 275 milliseconds (~40x slower)

Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams The update applies to the ~1,000 serverless endpoints we benchmark multiple times a day for speed, latency and caching. Key changes: ➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed. ➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts. ➤ Agentic workloads are now part of every measurement: we benchmark short, document-length and long-context prompts, plus multi-turn sessions across coding and knowledge work. Headline figures are the average across these workloads over the past 5 days. ➤ API default effort is now tracked alongside highest effort: most developers run models at the effort level each lab sets as its API default, so we now benchmark key models at that setting as well as at their highest effort. Effort names are not comparable across labs, so we use each lab's own default rather than one shared setting. Key takeaways: ➤ OpenAI models make up most of the Intelligence vs. Time per Task Pareto frontier despite slower output speeds: across effort levels, GPT-6 Astra and GPT-6.1 Sol hold the frontier from ~45 to ~55 on the Intelligence Index. At max effort, GPT-6 Astra runs at 51 tokens/s but finishes a task in ~17 minutes, against ~26 for Claude Opus 5.5 (max, 111 tokens/s) and ~29 minutes for Gemini 4 Carbon (high, 71 tokens/s), because it uses ~50k output tokens per task against ~120k to ~170k. ➤ API default effort cuts Time per Task by ~3 to 5x and Cost per Task by ~2 to 4x for GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5: all three default to medium effort, which scores ~5 to 11 points lower on the Intelligence Index in exchange. Claude Opus 5.5 falls from ~26 minutes and $10.04 per task at max to ~6.8 minutes and $2.45 at medium effort. ➤ Endpoints that optimize for decode speed alone see the largest drop in End-to-End Output Speed: high decode speeds can come at the cost of latency, for example by queuing requests to run at lower batch sizes, or by holding output and sending it in a single burst. Both lengthen the wait before the first token, which End-to-End Output Speed now counts.

Jensen Huang says engineers should burn $250,000 of AI a year, and Apple just cut its engineers' AI budget to about $50,000. And Apple plans to cut it again. Last month, some Apple engineers saw their daily AI spending cap drop from $300 to $200, Bloomberg reported. Some engineering teams could be cut to $100 a day next. Many groups outside engineering already sit at $100, and Apple has considered going as low as $25 for some of them. In March, Jensen said on the All-In podcast that he'd be deeply alarmed if a $500,000 engineer used less than $250,000 of AI tokens in a year. He also said Dario Amodei's forecast of hundreds of billions in AI revenue was too conservative. So Jensen's rule is simple: AI spending should run about half of what an engineer gets paid. By my math, at about 250 workdays a year, $200 a day comes to roughly $50,000. The planned $100 cap comes to roughly $25,000. By Jensen's rule, a $100 daily cap only makes sense for an engineer paid about $50,000 a year. And Apple still wants its people using MORE AI. It's steering teams toward cheaper models and AI that runs on the device, and Bloomberg's Mark Gurman has said Apple's internal work runs heavily on Anthropic's models. I think Apple just showed where the real limit on the AI boom sits now: The finance department. And that's just ONE company: - Accenture capped Claude at $29 a month per employee on October 1, after some staff had been spending up to $1,000 a month, 404 Media reported. - Uber burned through its entire 2026 AI budget in about 4 months, then capped AI coding tools at $1,500 a month per tool. - Microsoft has cut allowances too, and its AI chief Mustafa Suleyman told Bloomberg that Anthropic is extremely expensive and that Microsoft wants to eventually get rid of that cost. - Meta had internal leaderboards ranking staff by how many tokens they used, and now it has introduced token budgets. The usage is clearly there. Uber's engineers used the tools so much they spent a full year's budget in 4 months. So the people saying stop are the ones paying the bill. That's a huge problem for the AI labs, for one reason: Their spending is already locked in. Anthropic has committed to $895 billion, mostly for computing power. OpenAI has signed more than $1.4 trillion of compute deals. And paying for all of it depends on companies spending more on AI per worker every year. Apple, Accenture, Uber and Microsoft all pay for Anthropic's models, and Anthropic could start its IPO roadshow as early as next week. Its own filing shows revenue grew about 47% from the second quarter to the third, down from about 144% the quarter before, and price cuts took about $600 million out of revenue. Plenty of people will say this is just housekeeping, and they have numbers: Bloomberg says few Apple engineers ever used the full $300, so the savings may be small. Cheaper models can do the same work for less. Apple's R&D still rose 32% to $45.8 billion in fiscal 2026, partly because of AI. And more companies start using AI every month, so total spending can keep growing even with a cap on each worker. But Jensen's pitch depends on spending per worker going UP. Apple, Microsoft, Meta, Uber and Accenture are all writing rules that push it down. So the first question for Anthropic's roadshow should be simple: How much are its biggest customers spending per employee compared with a year ago, and how many of them have put caps in place? Who's right about what an engineer's AI is worth, Jensen Huang or Apple's finance team?

Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵