Climate and clean energy investor. Author of 5 books. Energy & Environment co-chair @SingularityU. Trying to build a better world.
Seattle
Joined May 2007
- Tweets70.8K
- Following10.2K
- Followers62.6K
- Likes21K
Pinned Tweet
Despite this election, I remain an optimist about America and the world. Humanity will continue to produce new ideas and new innovations to improve our lives. Good people will continue to come together to improve the world. And the political tide will turn. We'll make it so.
Rotten attitude. I disagree on the level of AI risk with many in the safety community. I also trust their integrity and their sincerity. And yeah, it's obvious that many in AI safety have sacrificed or made expensive choices due to their convictions. Respect.
Amen. AI is the most important new technology humanity has developed, perhaps ever. The last possible thing we should want is centralized control over it. Democratization and diversification of this technology is the path to resilience and freedom.
GE's first build of their 300MW BWRX-300 nuclear reactor, initially priced at an eye watering $4.5B USD ($15 / Watt !!), is behind schedule and over budget.
Comments:
1. This always happens with a new design. Always.
2. The only way nuclear ever gets cheap is to build the exact same design, over and over and over again. It's a chicken and egg problem. The first mover is at a disadvantage.
3. Calling this reactor an SMR is abuse of the term. Always has been. This is not a factory manufactured reactor. It's built in the field, with all the problems that brings. Just with a somewhat higher proportion of prefab parts.
Here we go again!
SMR Darlington Ontario neither on time nor budget
"Ontario Power Generation (OPG) has acknowledged that “DNNP Unit 1 is no longer expected to achieve the previously disclosed commercial operation date target of October 2030 or be completed within the previously disclosed estimated cost of $7.7 billion.”
OPG explicitly identifies unfinished engineering by GE Vernova Hitachi as the primary contributing factor going on to say…
“This reflects GE Vernova Hitachi (GEVH) Nuclear Energy’s ongoing engineering for its first-of-a-kind SMR, which has impacts on construction execution and supply chain performance.”"
decouple.media/p/opg-acknowl…
Very excited about this new AI institution. The top right quadrant they highlight here is pretty much where my views are. AI is powerful. Bottlenecks are real. We have agency. Congrats @RishiBommasani @sayashk @random_walker !
I am excited to announce our new nonprofit STABL, co-founded with the wonderful @sayashk @random_walker!
Our organization derives from four beliefs:
1. Advanced AI will broadly transform society and, if managed well, this transformation will be broadly beneficial.
2. The speed of this societal transformation is governed by bottlenecks: in many cases, the bottlenecks relate to people and organizations, not AI capabilities alone.
3. Since AI capabilities are not the sole determinant, the best strategy to manage the risks from advanced AI requires broad societal resilience, not just aligned models or capability delays, to provide defense in depth.
4. And while today’s institutions often struggle to grapple with the salvo of advanced AI, institutions will help stabilize society and will be essential, if adequately reformed, for ensuring shared prosperity.
In addition to these beliefs, the reason to create a new organization is based on the need we see in the ecosystem. In today’s ecosystem surrounding advanced AI, many of the leading companies and external actors focus on frontier AI as a technology. We share the view that the technology itself requires dedicated attention, and some of our work at STABL will be in this direction.
But the greater need we see is for independent organizations that take AI capabilities seriously and focus on downstream societal outcomes. We are building STABL to help fill this gap. And we hope to see many other organizations and initiatives emerge to define this field.
For me, the most exciting part thus far has been seeing all the support we have gotten in our early forays. Thanks especially to @audreyt @erikbryn @hlntnr @MichaelRStrain @RobbWiller for joining us as advisers!
We are hiring and fundraising so please reach out if you are interested in this work!
"based on what we expect from future models ... rather than on what we have observed from current models"
Anthropic amended their S-1 to say they have met their RSP AI R&D threshold of 2x acceleration that is "substantially attributable" to automation, based on what they expect from future models
15
1,616
Dario has a PhD in biophysics. Yet the below statements are just not credible. To me, this puts his judgment in question. Every time he says something like this, my confidence in everything else he says drops.
“Cancer, Alzheimer’s, heart disease, mental illness - these will be relics of the past in, I think, less than a decade.” — Anthropic CEO @DarioAmodei
What do you think, bio folks?
Source: instagram.com/reel/DfkundEE2…
To monitor AI, I'd argue that we don't need full mechanistic interpretability that can parse thoughts in detail. What we need are neural probes / classifiers to detect the categories of intent below. Probably a much easier task.
Most Anthropic progress acceleration is explained by acceleration of R&D spend. As discussed here: xitter.twitterapi.xyz/ramez/status/211956869…
Oddly ignored by AI Futures Project and mostly by the METR report.
100% agree. AI is hyper-competitive and default plural. No signs of natural monopolies or strong network effects. Most likely path to AI monopoly is through regulatory capture or poorly crafted policy. Much more here: rameznaam.com/p/plural-ai-ca…
Interesting and not surprising. If you constrain how long a model's chain of thought is allowed to be during post-training, you get models that can better control their chains of thought (making them harder to monitor).
Seems like we keep learning about ways RL can make models safer or less safe. I consider this a good thing. These lessons are valuable for the future.
I confess to some frustration when people give more weight to AI theories and models over what we see in the real world.
Synthetic benchmarks (METR time horizon) and theoretical models ("ECI is exponential") don't match the best empirical data we have. The theories are off rather dramatically when compared to the real world.
That real-world data is admittedly limited. We need more of it, with higher granularity, frequently updated. But in a conflict between your model and the best real world evidence that exists, choose the evidence. And press for more of that empirical data.
Conversely, if you don't update your models based on that evidence, why should anyone have confidence in those models?
Other AI models have moved capabilities forward as much or more than Fable 5.2. None triggered runaway RSI. Only ChatGPT o1 permanently changed AI improvement pace.
My hunch is that Fable 5.2 is a significantly smaller change to the AI world than o1 & reasoning models.
Very much agree. The METR report on progress at Anthropic is missing too much. I completely trust the METR team to go into this with integrity. But we all have different priors and sometimes pull very different conclusions out of the same data. I would ask Anthropic, as a start, to match the level of transparency that OpenAI provided in their September report.
The extent to which AIFP's updated forecast is so dramatically transformed by the recent METR report makes me think even more that one of the most important and beneficial things that Anthropic could do is release far more un-redacted information from the report immediately, or have the government force them to release more. (cc @KelseyTuoc)
It is possible some of the redactions are legitimate (and the most sensitive could still be limited), but I believe that the tradeoff of accelerating some of the lagging frontier is worth getting more clarity and consensus - the current public evidence base is just not enough to be able for the vast majority of elected leaders to justify urgent and extremely costly policy action.
The conclusions here are far too important to propagate if true, and it is simply not possible to convince skeptics by just saying trust METR.
We need far more people with different beliefs poring over these numbers in detail, because if what METR is saying is true, it is without exaggeration the most important and urgent story in the world right now.
Is AI progress accelerating? 🧵
1. Frontier AI rate of improvement on ECI has been fairly consistent. Fable 5.2's non-final ECI is a bit above trend. The big change is that Anthropic, in purple, has come from behind to take the lead.
Wherein I'm up front about my priors on AI RSI and ASI. And press for all of us to ground our models and forecasts on empirical data. Not hypotheses.
.@ramez argues that AI Futures Project's aggressive superintelligence forecasts may be shaped by assumptions its researchers held before deep learning even took off, rather than empirical evidence:
"11 years ago, I wrote a blog post saying the singularity is further than it appears because I was tired of all of the foom stories not taking into account diminishing returns. So I might be overly fixated on that past model."
"Any time you are trying to model something and you don't have empirical data to fit it against, you come in with your priors. There's like 100 choices you can make of exactly how part A relates to part B."
"A lot of people in AI, including the AI futurist folks, came in with a preexisting set of priors before deep learning even took off. They were reading Eliezer when Eliezer didn’t believe that machine learning was even a viable way to get to AGI."
"Our rational mind often is in service of the points we wanna make, and that’s okay. I’m gonna be straightforward about my biases. I'll do my best to update based on new data."
We need more emphasis on KYA - "Know Your AI" - to validate that the model you're running is indeed the one you believe it is, and to stress test it for adversarial behaviors.
