Onetime Reliability Engineer, now writer at MIRI. Opinions are my own.
Joined May 2024
- Tweets663
- Following51
- Followers132
- Likes537
Does anyone have theory of mind for what actually motivated Lutnick here?
Stunning report. At this point it seems like Howard Lutnick and Jeff Kessler are just outright defying the President's orders on export controls. The President apparently ordered Secretary Lutnick to crack down on China's access to controlled AI chips in Southeast Asia--and reportedly has ordered this three separate times--and they're just...not doing it.
Why is Lutnick refusing to act? Why is he allowing China to continue to get AI chips via Southeast Asia? At this point, Lutnick isn't just protecting Nvidia, he's straight up protecting Chinese companies--reportedly against the wishes of the President and the entire rest of the Cabinet. Is Lutnick personally benefitting from China getting access to AI chips?
This is a total breakdown inside the administration on a major national security issue. There has to be accountability for this. Congress obviously will investigate, but how is it that the President's own appointees are just refusing to execute the President's orders? I've never seen anything like this.
1
1
108
Teaching AIs to lie is still a horrible idea. Please do not hill-climb on this particular benchmark.
I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game).
The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies.
I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump.
Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game.
You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com
1
12
211
"But how could AI accomplish anything in the real world?"
i went to bed and asked claude to paint an autumn tree overnight.
at around 4am, it realised that it didn't have some shades -- so it stopped and titrated its own paint: diluting red, sienna and madder step by step on a second palette, then measuring each pocket from a photo.
woke up to this beautiful watercolor painting. 303 brush marks, took about 4.5 hours 🍂 (still working on the speed)
1
82
Win or lose, I'm glad to see we still live in a functional republic with peaceful transfer of power.
6
128
Another year, another dozen copies of The Phantom Tollbooth handed out to delighted trick-or-treaters.
What book should I add to the offerings next year? Taking suggestions; the current lead is The Last of the Really Great Whangdoodles by Julie Andrews.
1
48
People swear by all sorts of strategies for falling asleep. Counting up or down, flexing and relaxing muscles, deliberately hallucinating twelve phases of weird Twilight Zone stuff. They never work for me; my brain inevitably slides into ruminating about something or other.
The one thing that seems to work at least some of the time is imagining fight scenes, like from a movie or anime. The scene gets sort of increasingly disjoint and abstract and eventually I'm unconscious. Yes, this was a thing I did when I was like three, I probably anchored on it or something.
A less obvious connection: I wonder whether it has something to do with verbal vs nonverbal thought? It maybe seems easier to fall asleep if I can avoid an internal narration that can parse as actual words. A task which, as any meditator can observe, is harder than it sounds.
(Here "easier" means, like, half an hour instead of basically never.)
2
68
I recently encountered the following: "I haven't told my [parents/grandparents/family] that I think AI could kill us all soon, because they have enough to worry about already, and I don't want to upset them. There's not much they can do anyway."
Look. Maybe you know your family well, maybe your model of them says they'd rather be kept in ignorance. Maybe if you told them, they'd tell you they wished you hadn't. But if you aren't confident they would welcome that behavior from you, don't patronize them.
Instead, tell the truth to the people you love and let THEM decide what they want to do about it. Lay out your concerns frankly and clearly, and give them the same choice we all now face every day: call your reps, warn others, join a protest, take action -- or ignore it and get on with what life we have.
1
4
347
The case as I see it, stripped down to bare essentials: If we build something far superhuman at steering the future, we should expect the future to be dominated by whatever it steers towards.
In the special case where it robustly steers towards a thriving civilization of flourishing people, that is what the future will contain. In ~all other cases, it will contain something much worse, which likely does not include us. We die.
We do not currently have, and are not on track to have, a means to obtain the special case. What we do have is the capacity to say "stop" and enforce it, much as we did with nuclear weapons. This we must now do, or perish.
Palantir: "Don't worry, Europe! There's no possible way we could shut down the AI system managing your military data."
1
5
205
There will be many critical exploits in our future. We currently live in a tower made of holes.
AI systems found a working server takeover exploit in PHP, one of the most critical cybersecurity issues in recent memory!
- The vulnerability was found using a mix of OpenAI's GPT-5.6 Sol and Google's Gemini 4 Argon, plus human expertise
- PHP is the programming language behind ~70% of websites, most notably Wordpress
- The bugs sit in PHP itself, so nearly every PHP site is exposed no matter how well its own code is written. Merely running PHP itself is the risk
- The vulnerability lets an attacker with no login either run code on a server or read its memory, which can hold passwords, session tokens, API keys, or other secrets
- The flaw is from from how PHP cleans up when a request crashes after running out of memory: corrupted state is left behind and the next request can pick it up. An attacker can force that memory crash with a crafted upload, so even a bare page on a vulnerable version can be taken over.
In advance of midterms I've been getting swarmed with multiple political spam texts every day. The weird bit is that they all seem to think I'm still in Texas, when I left four years and two moves ago.
(Or sometimes Florida, but that one makes sense because of my phone plan.)
It's also strange to me that the spam filters aren't working better on these. They all follow a pretty obvious script. Is it really that hard?
1
55
Interp research suggests neurons represent individual concepts more often than previously thought.
This is a promising baby step towards eventually understanding machine intelligence. An international halt to frontier development might buy the field the time it needs.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
2
61
Vibe coding challenge I probably won't have time for: Mod Undertale so the combat system is turn-based or a puzzle or something, so my 70-something arthritic mom can enjoy the game.
I would go a step farther and say the answer to "can we align a smarter-than-human intelligence to any [specific] set of values" is a resounding "NO (with anything like current methods)."
The field does not look the way it would look if the answer were otherwise.
The “alignment *to whom*?” question IS important, but it creates the illusion that the primary questions in alignment are philosophical or political rather than scientific. The current high-order question in alignment is, “can you align a smarter-than-human intelligence *to any set of values*?” instead of “but whose values should it be aligned to?” I wish we had the luxury of having the rather more familiar political/philosophical debate of “to whom?,” but at this moment we do not.
OpenAI calls safety cases an "aspirational north star." Risk assessment professionals call them table stakes.
In safety as in regulation, OpenAI seems determined to set the ceiling where the floor should be.
1
2
204
Last year I wrote about why it's a bad idea to join a frontier AI company, even with good intentions. Today, Yoshua Bengio urges those who joined anyway to leave.
“If you truly prioritize safety, it is time for you to leave frontier AI companies.”
In an exclusive open letter to the researchers he trained — and the ones who trained on his work — AI godfather @Yoshua_Bengio calls on AI company employees to consider their own involvement in the rapid acceleration of artificial intelligence.
Read the letter in full, only on Transformer: transformernews.substack.com…
ALT Photograph of Yoshua Bengio. The headline reads: "Yoshua Bengio: 'If you prioritize safety, leave frontier AI companies'"
The technology for brain-computer interfaces seems much farther away than superhuman machines. But BCI could boost human potential and provide an edge for alignment research. It seems at least worth considering, for those willing to try.
The human brain is approaching its ChatGPT moment.
I spent 6 months writing Phase Lock: a manifesto on how brain computer interfaces can help align AI and preserve human agency.
If you only ever read one thing from me, read this.
3
93