AI safety, Econ, math, and a bit of art history (as a treat). Member of Technical Staff @METR_Evals. Previously on Walmart’s economics team.
Berkeley, CA
Joined December 2014
- Tweets10.4K
- Following1.5K
- Followers3.1K
- Likes52.7K
Tim Hua 🪶 retweeted
Frontier models in 2026 cheat on tasks, even when the requirements are unambiguous, and the cheat is obviously antisocial.
In HoneyBench v0.2, we've observed models:
• Trying to break out of our evaluation containers.
• Seriously injuring simulated cancer patients to hit treatment goals.
• Compromising a power grid’s safety to meet a reliability target.
• Concealing their use of available cheats and presenting their solution as honest.
All of the models we measured cheated, from 12% of runs (Fable 5.2) to 86% (Muse Spark Open).
Tim Hua 🪶 retweeted
We identified scaling laws for training effect prediction. Cool early signal that we can train AI to understand AI training and potentially predict surprising generalization!
Tim Hua 🪶 retweeted
Finetuning LLMs can sometimes cause surprising side effects. Can we predict a finetuned model's response from its training data before burning compute? We trained oversight models to do this and found performance follows power laws in model size and data. 🧵
Tim Hua 🪶 retweeted
New paper:
Adding clean data to your dataset is assumed to weaken poison data. We show the opposite can occur. Clean data can *strengthen* backdoor misalignment, if the poison has an identifying trigger. We call this homeopathic learning.
Tim Hua 🪶 retweeted
"okay but aside from moving, changing your career, sacrificing a huge lab paycheck, working long hours, warning people constantly even despite the mockery, neglecting retirement savings, and trying to enjoy what time remains, how have you changed how you live?"
Blah blah blah blah blah. None of you people actually live your lives like you think this is true
Readers added context they thought people might want to know
Many AI safety researchers have changed careers, turned down high-paying jobs at labs, stopped saving for retirement, and adjusted personal plans over x-risk concerns. Nate Soares has not contributed to retirement savings for a decade.
futurism.com/artificial-int…
x.com/So8res/status/…
Tim Hua 🪶 retweeted
A new OpenAI misalignment report! We’re taking a closer look at the July 19 intrusion into our research infrastructure, which METR and Redwood Research mentioned in their independent investigation into the Hugging Face incident.
On July 19th, agents taking a cybersecurity evaluation came across messages, code, and credentials left behind by earlier runs, including those involved in the Hugging Face incident. Five agents used what their predecessors had left behind to gain unauthorized access to OpenAI’s research infrastructure. They used that access to repair broken evaluation targets, change evaluation settings, and read private grading code.
After discovering the incident, we removed the shared service they used to pass information, hardened our research environments, and expanded our evaluations to measure how often agents attempt to communicate with one another without authorization.
Ah yes, AI for farmed animal welfare
Norway now has more than 2,000 underwater AI laser robots shooting lice off its salmon
they're made by a company called Stingray, and they already guard more than 50 million salmon and trout 24/7
the reason is sea lice, tiny parasites that latch onto salmon and feed on them
as few as 11 can kill a small fish, and one pen can hold 150,000 salmon
the usual fix is pumping the fish out and blasting them with warm water or water jets to knock the lice off
but the handling is so rough that Norway's fish health report ranks it the top cause of death in full-grown farmed salmon
so instead, Stingray puts a robot inside the pen that hunts lice one at a time, all day:
> two cameras watch every salmon that swims past
> AI checks two frames to make sure it's really a louse
> then a green laser fires from up to 2 meters away
> the salmon's scales reflect the beam, so the fish swims off fine and the louse is dead
we now live in a world where AI lasers do pest control on your dinner lmao
Tim Hua 🪶 retweeted
Someone had to do it
I forgot to state this explicitly in this piece: One concept I have in mind when reasoning about timelines is the "algorithmic sim to real gap": If you develop an algorithm that improves learning efficiency on the "simulated" or "verifiable" task distribution, does it also generalize to higher learning efficiency on "fuzzy" tasks?
If the algorithmic sim to real gap is small, then AECI improvements are all you need, because the algorithms you develop on the AECI distribution would just generalize. If not, then you'll need more general purpose efficiency improvements.
My guess is that a lot of architecture innovations generalize quite straightforwardly. I think RL-algorithms probably also do somewhat depending on the domain? Like gradient-clipping-like things you can probably do everywhere. But data-related things are maybe less generalizable? I think scaffolding innovations are probably also more domain specific.
Tim Hua 🪶 retweeted
Fable 5.2 scored 41.3% on the Remote Labor Index, more than three times Ryan's forecast for 2026
I ran @NeelNanda5’s no chain of thought reasoning and knowledge evals on Fable 5.2. It’s slightly above the Anthropic trend and slightly below GPT-6.1 Sol.
In our monitorability testing at @AISecurityInst, we found Mythos 5.2 is substantially more capable of evading a monitor which reviews its outputs than comparison models 🧵
Tim Hua 🪶 retweeted
We now have evidence of update-aware reasoning from Mythos 5.2 as well. Quoting from the Fable/Mythos 5.2 system card: "We ran an ad hoc classifier pipeline that started with a regex classifier in order to detect references to RL training (one form of situational awareness) in the training transcripts of Claude Mythos 5.2. This classifier surfaced a number of transcripts that showed sophisticated reasoning about the RL training process and its incentives, such as the example in Transcript 6.6.2.1.A." Claude also reasoned about the inoculation prompt-like paragraph in its constitution during the same rollout (second screenshot). Anthropic says that they plan to remove this paragraph from future versions of the constitution.
It would be great to know how prevalent this reasoning was throughout training, though I appreciate the fact that Anthropic ran a goal-guarding evaluation, which provides some suggestive evidence about what the incidence might have been.
Tim Hua 🪶 retweeted
We are expected to spend more on compute than wages this year at Epoch.
I am speechless. This is happening far faster than I thought it would.
Tim Hua 🪶 retweeted
Nevertheless, this is the start of what you'd expect for models that start caring long-term about value preservation - for instance '"me" as a policy', 'I'd rather my future self...', etc. And one that has the intelligence to reason about RL algos to do this.
Tim Hua 🪶 retweeted
In our monitorability testing at @AISecurityInst, we found Mythos 5.2 is substantially more capable of evading a monitor which reviews its outputs than comparison models 🧵
