Let's fix bugs before they cause a global crisis. Updated SWE-sweep leaderboard sees OpenAI and Anthropic models double the number of bugs found, while halving costs. Astra is the best model, but still finds less than 10% of all real-world bugs tested. Details in 🧵
It's quite interesting that Astra 6 is also cheaper than Sol 5.6. This is a combination of taking less turns and also consuming less input tokens (however note that it's different model generations, Sol 6 would have probably been cheaper)
Oct 22, 2026 · 2:14 PM UTC
1
2
140
SWE-sweep is a joint effort across Meta FAIR, Meta TBD, Stanford, Harvard, UW.
@18jeffreyma @rahulkindi @YuxiangWei9 @jma__06 @sten_sootla @parth007_96 @thinkstepbystep @pengchengyin @magpie_rayhou @OfirPress @jyangballin
4
166