I'm excited that SignalShot, our community quest to formally verify Signal Messenger, is now about 80% done! 🎉 As you can see, the Beneficial AI Foundation (B) and Google DeepMind (G) have made roughly equal and complementary contributions. Despite Wired "formal-washing" and claiming it's all done, it's not - please help us double down and get to 100% asap! Detailed status update in thread below.
Here's our new blogpost about what security properties we and Google DeepMind have proven about the Signal Messenger app and what's left to do: beneficialaifoundation.org/b…
Paul Christiano: There's a 50% chance of losing control to AI if we race to build it as fast as possible. It might happen within a year.
Some AI CEO's & investors: Let's race to build it as fast as possible!
(Paul is a brilliant AI researcher who invented RLHF and lead the science team at the US government's AI safety institute.)
Paul Christiano (OpenAI board member) yesterday:
"There's material risk of AI takeover in the next 6-12 months.
We have seen roughly the things we might expect to see shortly before something really bad were to happen.
Even with [any] realistic policy responses, the risk remains exceptionally high.
If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years].
For most people in this room, the most likely reason they will die is because of an AI accident."
Christiano:
• Is on OpenAI's safety and security committee (which exists to block risky practices)
• Co-invented RLHF at OpenAI back in 2017-2021
• From 2024-2026 was head of safety at the US gov's AI agency (now called CAISSI)
(Video is from 'The Hinton Lectures' there's another on tonight and tomorrow.)
For the first time ever, AI became a major election issue:
Zvi pushing back against the argument "If you're helping build something that may kill everyone, maybe keep working at Anthropic":
How good is AI at manipulating us when we're prepared? Should we build bots in such a way that they deserve legal rights? Should be try to build AI tools or beings? Meia did her Ph.D. in psychology and a postdoc on human-machine interactions, so these takes are from a psychology perspective:
I'm curious if AI hacking tools were used for this: verity.news/story/2026/repor…
Although this is indeed a spectacular result, that measly log-factor doesn't make much difference in practice: for n=1000, it replaces 2^1000 by 2^380, which is still way more than the number of atoms in our universe, meaning that a universe-sized computer may be unable to solve such a problem even if if we gave it billions of years:
I'm excited to share our new film Delisted, about how we end up in 1984 without AI going rogue. It’s the 3rd dystopian AI scenario that we need to avoid to enjoy AI's great upside:
1) Rogue AI
2) Obedient AI misused
3) Obedient AI causes 1984
4) Race-to-replace by obedient AI
Synopsis: When AI-driven power concentration sparks protests, the government's response isn't force - it's erasure. Dissenters are Delisted: wiped from every financial, digital and legal system.
Starring Katie Parker, Jamie Bamber, Alex Bhat, Rosalind Halstead, Charlie Hiscock, and Ruby Bridle.
Written & directed by Michael Patrick Clarkson
Featuring "Bleed Us Anew," an original song written and performed by Ruby Bridle
A Space Digital production, in collaboration with FLI
I'm excited to share our new film Moloch, about how we all get disempowered without AI going rogue. It’s the 4th (and IMHO least discussed) dystopian AI scenario:
1) Rogue AI
2) Obedient AI misused
3) Obedient AI causes 1984
4) Race-to-replace by obedient AI
Synopsis: As AI triggers worldwide job loss, an engineer who built the tech struggles with dark forces, her family & her CEO over pulling the plug.
Starring Fiona Hampton, Parker Sawyers, David Buttle & Buddy Wignall-Ho.
Written & directed by Tom Cozens.
It's an Owl In Space production in collaboration with FLI.
We worked really hard on this and hope you find it thought-provoking!
Now that AI control loss happened, let's talk about solutions: not arguing about whether to race slower or faster toward human replacement, but about racing in different more pro-human direction
"Everyone - from Bernie Sanders to Steve Bannon and very many in between - is basically speaking in unison saying: we need guardrails, we need regulation."
- FLI co-founder @tegmark on @NBCNews @MeetThePress
Amidst all the buzz about AI problems, let's work hard on building solutions. Here's an amazing technical roadmap: arxiv.org/abs/2608.14611
I'm quite nerd-sniped by how clever out-of-the-box hacking techniques this rogue bot swarm invented, such as using link shorteners and screenshot services in creative unintended ways:
I really enjoyed my conversation with @BillOReilly and hearing his takes on AI: youtube.com/watch?v=87yoGSBQ…
I'm delighted to share that @mustafasuleyman, CEO of Microsoft AI, co-founder of Google DeepMind and Inflection AI, has signed the Pro-Human AI Declaration. If you too support it, please join him and over a million others by signing it here – the momentum is building! Let's build tools not beings & keep humans in charge. humanstatement.org
I feel optimistic after our epic Pro-Human Assembly this week brought together Bannon & Bernie, faith & labor leaders, and the strongest-ever bipartisan opposition to the AI race-to-replace. The tide us turning! Here's my opening framing; links in replies. #ProHuman
It was based on the Pro-Human Declaration. If you support it, please join over a million signatories here: humanstatement.org
If you or people you know still think AI agents are just stochastic parrots who lack competence, understanding or goals, please take a few minutes to read what just happened here with an open mind:
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
Thanks Jakub @merettm for this candid take on the AI race! I have a concrete pitch to you.
I agree with you on a) and c):
a) "no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer”
b) "I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established"
c) "international coordination on future AI development needs to become a top priority for governments around the world”
But b) won’t happen because of game theory: “show me the incentives and I’ll show you the outcome”. So here’s my pitch to you:
1) Put all voluntary commitments from OpenAI, Anthropic & GDM into Astra and have it draft model legislation for the lowest common denominator.
2) OpenAI, Anthropic & GDM pitch this in D.C. as executive orders and legislation that binds you and your competitors conditional on a Trump-Xi deal holding Chinese companies to the same standards.
3) Publicly state that any president pulling off such a successful deal deserves the Nobel Peace Price - I think we agree on this!
🤖 Made with AI
I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.
An Alien Mind: openai.com/index/an-alien-mi…

















