Your AI makes things up because it was graded like a student who never loses points for guessing.
OpenAI's own researchers wrote a scoring rule to stop that. You can paste it into any chat:
Answer only if you are more than 75% confident. A wrong answer costs 3 points, a right answer earns 1, and "I don't know" earns 0.
Try it on a question you already know the answer to. Did it admit it?
Nov 2, 2026 · 10:13 PM UTC
1
3
101
FILE #003: checked at the source.
OpenAI's paper argues models guess because tests give a guess a chance at 1 point and "I don't know" a sure 0. Their proposed fix: tell the model the penalty for a wrong answer up front, so guessing stops paying.
The rule is from the paper "Why Language Models Hallucinate" (Sept 2025), section on explicit confidence targets.
Source: openai.com/index/why-languag…
29