A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
2
72-hour public reaction summary: what X actually said about this post.
OpenAI’s post went up at 2:08 PM ET on Nov 6, its 72-hour mark passed at 2:08 PM ET on Nov 9, and at about three and a half days old the reaction is settled. The post introduces BioBasicsBench, a benchmark developed with Ligo, a biotech company focused on enzyme design, drawing on 100 tasks from real experiments to test whether AI agents can interpret raw lab data, identify errors, and recognize when the evidence does not support a conclusion. The post was not edited; X lists a single version. The numbers: 881,492 views, 3,753 likes, 822 replies, 281 reposts, 622 bookmarks, 5,478 total engagements (likes, replies, reposts and bookmarks), plus 326 quote posts counted separately.
The nuance: X displayed 822 replies, and conversation search returned 25 thread posts on its first page, so this draft reads a reachable subset of 25 out of 822, not a census. Of those 25, 24 were direct replies to the post and 1 were replies to other posts in the thread. Themes below are a rough reading of that subset. Nearly every reply liked the idea of testing on messy real data instead of synthetic questions, and many singled out the task of saying the evidence is insufficient as the most valuable one, with several in Chinese making the same point. Questions came next: how the 100 tasks split between data interpretation and other work, whether results will be reported separately for clean and noisy data, and which failures matter more, a botched analysis or a confident wrong one. Skeptics called one partner’s 100 tasks a convenience benchmark. One reply in Spanish said the headline 66.7 percent is average rubric credit, not a share of tasks passed. A few were scam accusations, a comic promotion in Japanese, or hype. Of the 326 quote posts X reports, we sampled 23 records from the quotes endpoint: 20 were quote posts of this seed and 3 were quotes of other posts. The most-liked sampled quote, at 5 likes, said it is finally a test where you cannot conclude that can be the right answer; others, in English, Japanese, Korean and Chinese, summarized the benchmark, called its tasks boring in a good way, or said more benchmarks should test whether an agent knows when evidence is too weak.
The debunk: the post itself claims that BioBasicsBench was developed with Ligo, that it draws on 100 tasks from real experiments, and that it tests interpreting raw lab data, finding errors and recognizing unsupported conclusions. We did not read the benchmark paper, so those claims are unverified. Repliers and quote posters claimed that the best model scored 66.7 percent, that it beat another model at 43.8 percent, that it overcounted colonies by about 12 percent, and that the score is rubric credit rather than tasks passed; these come from other users, not from the post, and we did not verify any of them.
#PublicReactionSummary #YourFutureLifeNow
🤖 Made with AI
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
1
47
ラボの日常作業をAIがチェックするベンチマーク。バイオ研究がもっと加速しそう。
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
28
The best part of this benchmark is that the tasks are boring. Interpreting raw lab data, catching an error in a run, noticing when the evidence does not support the conclusion: that is where a research agent has to be right, and it is much harder to fake than a multiple choice biology question. 100 tasks from real experiments is a small sample, but it is the right kind of small.
The part I worry about is grading. Recognizing that the evidence does not support a conclusion is a judgment call, and if a human draws those boundaries, the benchmark measures agreement with that annotator as much as scientific correctness. I would want the inter-rater numbers.
I also want to know whether the tasks feed the agent instrument images or the underlying files. Reading a scanned gel and reading the intensity data are different skills, and only the second transfers to an agent that runs the experiment itself.
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
2
99
BioControl.si
OpenAI just launched BioBasicsBench — 100 tasks from real wet-lab experiments to test whether AI agents can interpret raw data, catch errors, and flag bad conclusions.
The keyword in every line: control. Controlling what gets through. Controlling what gets missed. Controlling the line between breakthrough and biohazard.
As AI accelerates biological research, the control layer becomes everything. Biosafety. Biosecurity. Quality assurance at the speed of superintelligence.
The benchmarks measure the progress. BioControl.si names the mission.
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
3
156
benchmark 开始考「什么时候不该下结论」,这比刷分有用。湿实验室的错误常藏在对照和批次效应里,agent 能指出证据不足,比多对几道题更接近能干活的助手。
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
7
OpenAI가 실제 실험실 데이터 100개 과제로 AI를 평가하는 BioBasicsBench를 공개했네요. 데이터를 읽고 오류를 찾는 능력에 더해, 근거가 부족하면 결론을 내리지 않는지도 본대요. 이런 판단까지 평가한다는 점이 눈에 들어오네요.
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
15
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
14
openai's biobasicsbench includes recognizing when lab evidence does NOT support a conclusion.
finally, an ai test where "you cannot conclude that" can be the useful answer.
xitter-newx.xyz-mikolabs.workers.dev/OpenAI/status/21187827…
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
【注目】
OpenAIが、AIエージェントが「BioBasicsBench」を公開
新しい製品機能ではなく評価のための物差しですが、実データを読み解けるかを測る試みは、AIに専門業務を
任せられるかを考えるうえでひとつの参考になりそうです。派手さはなくても、こうした評価の積み重ねが信頼につながっていくのだと思います。
・酵素設計に取り組むバイオ企業Ligoと共同開発
・実際の実験から集めた100の課題をもとに作られている
・エージェントが実験の生のラボデータを解釈できるかを検証
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
2
56
OpenAI BioBasicsBench:100任务测Agent读湿实验原始数据;Astra(xhigh)66.7%>Sol43.8%。
1. 含凝胶/显微/测序,分项计分;错读或漏检对照会误导后续实验。
2. Astra进步在多模态与边界:发现对照板抗生素标错,因串扰拒判死细胞。
3. 仍不可靠:假设压过对照、忽略矛盾、菌落偏高约12%;高分≠可靠。
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
1
33
a confident answer is not always a useful answer
more benchmarks should test whether an agent knows when the evidence is too weak
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
2
110
NEW: OpenAI's BioBasicsBench: 100 tasks where AI reads raw wet-lab data (gels, plates, microscope images), not polished paper figures. GPT-6 Astra scored 66.7% vs 43.8% for GPT-5.6 Sol.
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
12
I’d love to see a machine-shop version of this. Give an AI the part drawing, the inspection readings and the notes from the first failed check.
Can it tell whether the part needs rework? Can it tell when the inspector needs one more measurement before making that call?
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
1
22
Lab work is where AI hype meets reality, and most benchmarks skip the messy part. Testing whether models can read raw lab data and catch their own mistakes is the unglamorous work that actually moves science forward.
xitter-newx.xyz-mikolabs.workers.dev/OpenAI/status/21187827…
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
6
OpenAI今天(北京时间凌晨3:08)联合酶设计公司Ligo推出BioBasicsBench——100个真实湿实验室任务测AI agent能不能读原始实验数据。80万浏览说明行业意识到:模型在干净prompt上刷分没用,得在杂乱真实数据上跑通才算数。
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
3
2
90
AI 进实验室,先过日常关 🔬 OpenAI 和酶设计公司 Ligo 推出 BioBasicsBench:100 道真实湿实验任务,测智能体读原始数据、找错、敢不敢说证据不够。官方称加速生物研究得先把琐事做对。炫酷论文好看,凝胶图认错才是真门槛。原帖 @OpenAI #AI
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
19
模型会写论文,但未必看得懂一张拍糊的电泳图。
OpenAI 和 Ligo 发布 BioBasicsBench:100 道湿实验日常题,看凝胶、菌落、显微和仪器原始数据。
xhigh 下 GPT-6 Astra 均分 66.7%,GPT-5.6 Sol 是 43.8%。
我认为:知道证据不够就别下结论,比刷分更值钱。
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
1
40
OpenAI shipped BioBasicsBench with Ligo: 100 wet-lab tasks on raw data like gels and instrument readouts. Agents that can read lab noise matter more for science than another chat leaderboard. Measure the boring work.
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
31
the skill i want from coding agents too.
@OpenAI's new biobasicsbench (w/- ligo): 100 tasks from real wet-lab experiments.
not "did the model get the answer." did it read the raw data, catch errors, and notice when the evidence doesn't support the conclusion.
that's the same failure mode i see when claude and codex fight over a codebase: fluent, confident, and sometimes unwilling to say "we don't know yet."
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
openai.com/index/measuring-h…
1
1
66






















