Below is my personal opinion, not speaking for openai.
There are many AI safety interventions that are purely positive.
Training for honesty, and alignment in general, is one example. Models being more honest & aligned is undeniably a good thing.
Monitoring is another example: it serves both to block harmful actions as well as learn valuable information about misaligned behavior. (And to me, the fact that it comes at a non-trivial compute cost, and so has some slowdown impact, is a feature, not a bug.)
Nerfing or restricting models from helping with AI research is more complex. I can see the argument for preventing bad actors, in particular ones from non democratic nations, from creating models with risky capabilities.
However, I am also worried about concentration of power, in particular concentration of power by us (= the labs), and this seems to go in the opposite direction. It can also hamper external AI safety researchers. And to the extent a lot of the risk in acceleration is from internal deployment, having growing gaps in capabilities between internal and external models seems not ideal.
I am not saying that there will never be a good reason for having some restrictions in some form, but this is an area we should tread *very* carefully on.
There is also something I find problematic about even the perception of using safety as an excuse for practices that may also be motivated by competitive reasons. It is fine that for-profit corporations try to optimize their profit, but let's make sure we're honest about it and not hiding behind safety.
As models become more capable, securing internal deployment is more important than ever. Control Room gives us the ability to track, monitor, and stop jobs across our research infrastructure.
We built Control Room, a central system to track model workloads across our research infrastructure and stop them quickly when safety or security concerns arise.
It connects every research inference workload, including agentic sampling for model training and evaluation, data processing, and more. This works together with our misalignment monitoring systems, and halts model output within 35 seconds of a decision to intervene to stop or pause a job.
Control Room is just one part of our holistic approach for safe frontier model training and deployment.
openai.com/index/control-roo…
An update on my AI safety course with links to some videos and scribe notes.
lesswrong.com/posts/WsTiYqDb…
Sounds like a throwback to the types of “beefs” people had with mathematicians way back in July.
I have my own criticisms of Anthropic but I respect the fact that Amodei gave significant amounts to charity well before he was wealthy, and it’s unfair to criticize him on how he reasoned about how to allocate his own money.
One of the challenges in using models for research is that models sometimes refuse or underperform tasks that seem "fishy" or raise ethical challenges, but often it is precisely the safety research that requires "fishy" tasks.
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
The model spec has multiple audiences: the model, us, developers, users.
But the person that reads it most carefully is Zvi.
Quick comment on Scott Alexander's SURA essay.
I agree with several parts of it, but there is a huge spectrum between "stop unilaterally" and "race to RSI at maximum speed, then try to solve alignment with right-on-the-brink-of-superintelligence AIs."
It also seems to imagine that the release of the aligned super intelligence is basically the end of history, and is very unclear on what happens after this "last possible moment." Section IV is not very reassuring.
Link to essay lesswrong.com/posts/xikGZ8fu…
Also I realize Section IV talks a bit about "the day after" but it's not very encouraging and seems to assume that the people building AIs will be in control and it would be up to them if to hand it to government or keep it.
This is smart, honest and points to a much bigger issue than the CEA. If we want a future with humans in charge, we need to find ways to train people for that, but AI upends the incentive structure that used to work for apprenticeships
My comment to @neilirwin about 20 CEA staff being replaceable by AI generated discussion. My followup essay on the issue & what it means for the future of work, training and the pipeline.
We shouldn't avert our gaze from the tough issues we face. econvitals.org/posts/2026-11…
This video is great, but in case people want a musical version asked codex to orchestrate Claude and suno to make it. ChatGPT and Claude get along quite well 😉
youtu.be/5SdTrybDhaI?si=xLGt…
An easier way to explain why e is natural: video finally released! youtu.be/03n3SQBAUrs
I actually recorded this last year. But none of us had time to edit the footage. Then I built a full video editing pipeline in Claude Code. Amazing what can be done now! (All real footage)
Dana’s description of the proof.
Talk by Dana Moshkovitz on the proof of the Unique Games Conjecture.
scottaaronson.blog/?p=10232
Talk by Dana Moshkovitz on the proof of the Unique Games Conjecture.
scottaaronson.blog/?p=10232
We are monitoring our frontier models for misalignment across a wide array of deployments, including not just coding agent use, but also training and evaluation.
This enables detecting and investigating incidents (see our misalignment reports!) and also paging humans in urgent cases.
Some details on what our monitoring system currently looks like openai.com/index/how-we-dete…
We organized the Model Spec around five core values:
1. Be faithful to human intent.
2. Minimize harm.
3. Be honest and transparent.
4. Facilitate human agency.
5. Act with care.
We’ve updated the Model Spec, our public description of how we want our models to behave. This revision makes clearer what we want from models as they take on more autonomous work in the world and become capable of helping us in new ways. 🧵
openai.com/index/updated-mod…
This Perelman lab seems to have pretty strong models. Maybe I should try them out. How good are they in computer use?
what a time to be alive
all seven Millennium Prize problems have now seen breakthroughs or related advances of some kind:
P vs NP: ETH disproved; UGC proof announced
Riemann Hypothesis: quasi-RH proved and formalized; Landau–Siegel zeros excluded
Yang–Mills: existence proof announced and formalized
Navier–Stokes: counterexample announced and formalized
Hodge Conjecture: partial proof for CM-type abelian varieties
Birch–Swinnerton-Dyer: full formula proved for selmer coranks 0 and 1
Poincaré Conjecture: solved by Perelman in 2003
This is really cool work, showing how "AI advisors" can bias the decisions of those that they are advising.
Beautiful overview of how work changed, from the weavers of the 1760s to the mathematicians of 2026.
"The goal of this essay is to present a historical record of technological change and the anxieties around it, not to propose a solution to today’s concerns."
New Substack post with @soumitrashukla9. Work has never been just about the income earned sans effort costs, as typical in economic models. Key aspects of people’s identity, their meaning, and sense of agency are all wrapped up with what they do for a living.
There has been a lot of conversation lately about how technological change, particularly AI, may disrupt this relationship, from Ezra Klein's column on his own mathematician father’s grapplings with losing meaning in mathematics, to Francis Fukuyama's post on AI, agency, and recognition. In this piece we take a historical perspective on how technological transformation has disrupted identity and meaning by affecting jobs, and how workers and society responded to these changes.
Much of the debate around AI and jobs has centered on wages and material consequences, but losing one's work is also intertwined with how people view their place in society. Most people know about the Luddites—skilled weavers and knitters—who protested the devaluation of their mastery. What’s less known is that the Luddites were actually experts in using the machines as well, and that their protests prompted prominent economists like David Ricardo to rethink the role of automation in society, and later inspired the Arts and Crafts movement. A century later, when recorded sound displaced 20,000 cinema musicians, their campaign against "canned music" and two-year recording ban pressured labels into funding live performance.
Soumitra and I have been working on versions of this essay for months. He led the piece, and we’re both really excited to get it out there.
Link to the post here: aleximas.substack.com/p/tech…
I don't know what's the problem. Zitron was absolutely correct in saying "This company is not worth $60 billion."
Some entertaining highlights from wheresyoured.at/wheres-the-m… 20 months ago:
- "Anthropic has projected (made up) that it will make at least $12 billion in revenue in 2027"
- "This company is not worth $60 billion. Anthropic has raised $14.7 billion to create an also-ran Large Language Model company"
- "Anthropic, much like OpenAI, has categorically failed to productize its Large Language Model"
- "Its CEO is also a sleazy conman who, like Sam Altman, continually promises that his company's AI systems will become powerful and autonomous in a way that they have never shown any possibility of becoming."
- "Any investor in Anthropic needs to seriously consider what it is they're investing in. Its products are both unpopular and commoditized, and it lost $5.6 billion last year! Stop dancing around this fact! Stop it!"
- "there’s a lot of demand to put AI in stuff and some demand to buy stuff with AI on it by enterprises buying software, but little evidence to suggest significant user adoption or usage, I’d argue because Large Language Models do not lend themselves to features that provide meaningful business returns."
- "When a company uses the term “agent,” they are intentionally trying to be deceitful, because the term “agent” means “autonomous AI that does stuff without you touching it.” The problem with this definition is that everybody has used it to refer to “a chatbot that can do some things while connected to a database,” which is otherwise known as a chatbot."
- "It isn’t working. The users aren’t there. The revenue isn’t there. The best time to stop this was two years ago, and the next best time is as soon as humanly possible. I have said that generative AI is a group delusion in the past, and I repeat that claim today."
- "we are approaching the limit of what the transformer-based architecture can do, if we haven’t already reached it. No amount of beating off about test-time compute and connecting Large Language Models to other Large Language Models is going to create a new use case for this technology"
- "generative AI is OpenAI. The consumer adoption of this software has completely failed, and appears to be going nowhere fast. ChatGPT is sustained entirely on deranged, specious hype drummed up by a media industry"
... and my favorite ...
- "I will keep writing this stuff until I’m proven wrong."
While mathematics and TCS share a lot in common, it is up to each community to decide how best they want to address the question of AI's impact on their field.
Our letter to the community invites ideas for organizing this effort and, separately, for addressing the broader questions raised by AI-generated results:
docs.google.com/document/d/1…
OpenAI's announcement is here: openai.com/index/building-sc…
