Founder of @1daysooner, @rikersdebate & @waitlistzero.
Brooklyn
Joined August 2021
- Tweets6.7K
- Following1.9K
- Followers1.1K
- Likes24.3K
Ths was a kind of neat experiment -- the early, chaos section held my attention better than the latter parts which felt pretty trite and boring.
I asked fable 5.2 to invent a new musical form.
Fable 5.2 wrote a symphony that’s shaped like a training run.
The orchestra starts as random noise, learns the key, pulse, harmony and the melody, then plays music it was never shown.
115
Josh Morrison retweeted
This newsletter is my new favorite source for easily following AI developments: p3humansonai.substack.com/p/… props to @peligrietzer et al
Josh Morrison retweeted
I don't understand why people are taking comfort from this. Yes, capabilities grow sublinearly as you increase dollars. But dollars are growing way super-linearly, with lots of room to keep growing -- Anthropic could continue its previous ~5-6x scale-up pace for at least another year and maybe longer if revenue keeps growing at its crazy clip. And "another year" is enough to get to extremely dangerous tech if the METR model is right.
AI progress per dollar of investment shows diminishing returns (as we've all known, right?) This is using METR's estimate of acceleration at Anthropic. That acceleration isn't enough to overcome the diminishing returns. by @sebkrier
Josh Morrison retweeted
The best time to scrap the filibuster is when there’s divided government and the only thing that changes is random message bills get vetoed instead of dying — it makes the change less scary and dramatic and just sets up clean legislating for the future rather than as a crisis.
Josh Morrison retweeted
The Wiles-to-Miller switch bodes well for Robert F. Kennedy Jr., who looks increasingly likely to stick around as health secretary instead of running for president again. washingtonsun.com/health/rfk…
Josh Morrison retweeted
Getting a lot of criticism that's based on obviously incorrect premises about your motivations or basic facts of the situation is subtly corrupting, it trains you to dismiss criticism and makes it pretty hard for people to get through to you on things you really are wrong about
Josh Morrison retweeted
And if we’re serious about improving healthcare, can we please compensate kidney donors, which could reverse stagnant donation rates and help save the ~10k Americans who die every year waiting for a kidney?
There are 47 cosponsors on a bipartisan bill that would do just that:
Josh Morrison retweeted
Here’s a pretty important for thing for a new Dem congress to get to work on right away!
This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic's central plan is to continue this acceleration of AI capabilities until we're getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
Josh Morrison retweeted
Clinical trials in the US are slow, costly, and often out of reach for patients who might benefit from participation. 1Day Sooner and 18 organizations, 13 of them rare disease groups, are backing the bipartisan Faster Clinical Trials Act from @RepAuchincloss and @RepMGriffith.
Infinita is proudly declaring its support for the "Faster Clinical Trials Act", a bipartisan bill introduced by Reps.
@JakeAuch (D-MA) and @RepMGriffith (R-VA).
Clinical trial reform at the federal level has suffered from a lack of ambition.
This is the first bill that I've seen that is not lacking ambition whatsoever.
"The bill also sets targets to reduce the cost per patient for the clinical trials conducted by these networks by 30 to 50 percent and speed participant enrollment in those trials by 30 percent."
The mechanisms how to reduce costs are a distillation best policy research that I've seen over the years, incl. by @IFP, @RuxandraTeslo, @sytses & many more
The things I am most excited about:
- Bringing evidence generation closer into the clinic; proving real-world effectiveness has been an afterthought in FDA approvals, leading to lack of safety & efficacy
- Streamlined trial site networks that can be accessed via a "multi-site, single-IRB", overcoming the IRB fragmentation that is a major source of delay in early research
- Governance sandboxes - critically, the bill does not make the mistake of trying to fix everything at once. Its mandate is to support sandboxes, innovative trial models
With this, the US has a real shot at retaining its competitive advantage in biotech, and could do right by millions of patients to access trials in the US instead of abroad.
Josh Morrison retweeted
Our colleagues found that Anthropic’s capabilities progress has doubled, mainly due to AI. Ryan’s modeling from public info suggests the whole company’s R&D would be going 4-5x faster. Labor uplift would be higher still. This is a major update to very short timelines.
This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.
I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.
I haven't seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.) Less extensive redactions would greatly reduce this need for trust.
Since in my current role I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now. Here, I’ve done some analysis of the implications using my previous public statements.
A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027.
Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger:
- Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress.
- Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred.
We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed (lesswrong.com/posts/FG54euEA…) and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 3 * 2^(~0.6) = ~4.5.
(Relevant public modeling and numbers I'm using: lesswrong.com/posts/FG54euEA…, lesswrong.com/posts/Jga7PHMz…)
Means Dems will likely control the Senate for the next six years and almost definitely till at least 2030
2
196
Josh Morrison retweeted
Can data from health records and insurance claims make medical research faster, cheaper, and more informative?
Join us Monday, Nov 9 at 10:30 AM ET for a live expert panel on real-world evidence in clinical research and regulatory decision-making.
Josh Morrison retweeted
More than 10 years since Hep C cures, HHS OIG finds just 23% of Medicaid beneficiaries received treatment within 6 months of testing! Treatment rates were even lower among newly infected beneficiaries, dual-eligibles, older adults, and children.
oig.hhs.gov/documents/evalua…
Josh Morrison retweeted
For the past few months, @ReadTransformer has been working with the incredible team at @NYMag on this: The AI Power List.
When NY Mag first reached out about collaborating on this, I was thrilled. Everyone knows the big names in AI: Sam, Dario, Demis, Elon, Jensen and the like. But so much of the *real* power in AI is inscrutable to the wider world. It is held by people well-known to insiders, but with little-to-no public presence — despite their outsized influence on the future of humanity.
NY Mag and Transformer profiled 26 of these powerful insiders. They're the people deciding how AI is used, developed, regulated, and thought about. They're remarkably intertwined, universally fascinating, and still have everything to play for.
These new power brokers will decide how AI shakes out, refashioning the world for the rest of us in the process.
It's time to get to know them.
AI has minted a new power structure. A handful of private companies command the might and wealth of small nation-states and are mostly left alone to self-govern. A few hundred employees are on the brink of becoming centimillionaires and considering which causes and presidential candidates they will funnel their vast resources toward.
One way to tell the story of how AI is changing our world is through some newly familiar names: Sam Altman, Dario Amodei, Jensen Huang. But an astonishing amount of power rests in the hands of a few dozen other people, some virtually anonymous.
For this year’s annual “Power” issue, we are surfacing 26 people whose influence on the AI landscape far outstrips their fame.
We partnered with @ReadTransformer, a publication with a command of the politics within transformative AI, whose team drew on its years of reporting, as well as new interviews with more than 50 high-ranking industry leaders, to help articulate what makes these people powerful.
The AI scene is small. Many are remarkably intertwined, sharing offices, homes, and funding. The list’s average age is 36. It is also overwhelmingly male. Read about the few dozen power brokers who will decide how AI shakes out, refashioning the world for the rest of us in the process: nymag.visitlink.me/AgEtqI
Josh Morrison retweeted
Agree with Boaz and wanted to expand on a couple points.
@Slatestarcodex's SURA essay has many good points and i'm glad he wrote it, but I didn't like how much it sometimes conflates the idea that companies should totally stop AI development unilaterally with the idea that companies should exercise more unilateral restraint that might slow them down/hurt their relative position to help us collectively get to a better equilibrium. (and that they should be investing far more resources on solving the collective action problem through e.g. support for policies that could actually effectively restrain them and their competitors).
I worry that people arguing against ~any costly unilateral constraint at safety motivated companies will cite this essay as supporting their views even if that isn't Scott's intent.
To put it another way - there is a difference between saying a country should get rid of all their nukes unilaterally vs whether they should accelerate development of larger hydrogen bombs and hypersonic missiles and not really particularly prioritize even setting up multilateral nonproliferation talks.
As a bottom line: I think that there are situations where continuing to race to plausibly catastrophic results is defensible for the sorts of reasons Scott lays out, but the bar towards this being the strategy should be extremely high and rely on making clear to everyone around you that you would love to accept an offramp to get away from the current cursed equillibrium. The bar is even higher if you are not just trying to remain competitive in the race to have a seat at the table in future negotations, but actively pushing the limits and accelerating in particular the kinds of development you believe to be most dangerous.
(Of course, one reason among many why this strategy is so questionable both morally and pragmatically, is that it is so incredibly unintuitive to external observers that it will lead them to doubt your claims about the relevant risks.)
Quick comment on Scott Alexander's SURA essay.
I agree with several parts of it, but there is a huge spectrum between "stop unilaterally" and "race to RSI at maximum speed, then try to solve alignment with right-on-the-brink-of-superintelligence AIs."
It also seems to imagine that the release of the aligned super intelligence is basically the end of history, and is very unclear on what happens after this "last possible moment." Section IV is not very reassuring.
Josh Morrison retweeted
for the record i pretty much completely agree with scott and think this is an extremely important post. i strongly think the *sane*, *safe* path forwards is via some form of treaty and enforced global pacing of ai development. but in fact the *majority* of my probability mass for humanity surviving still lands on *one of openai or anthropic solving alignment*. i think this is extremely difficult, very risky overall, civilizationally a pretty baffling and stupid thing to do, but i also think it has quite a ~large chance of success. i'm very very happy with the recent societal shifts that have increased the chances we get global coordination on this rather than having to rely on technical solutions under terrible dynamics and time pressure. but i find the arguments people tend to present for things like unilateral pause or encouraging every lab employee to quit immediately because they're doing something horrific pretty much totally unconvincing.
lesswrong.com/posts/xikGZ8fu…
Scott Alexander: "I... made fun of our misinformed opponents, who insisted that if we unilaterally paused AI then China would win. But actually, I said, nobody wants to unilaterally pause AI; this is misinformation... Here I am forced to eat crow"
lesswrong.com/posts/xikGZ8fu…
This is an underrated policy success and frankly the result of European regulation
xAI announced today it is adding text watermarking to Grok. This appears to be rolling out globally. xAI was the last holdout, so the only model left that (as far as I know) does not watermark text is ChatGPT, and only ChatGPT in North America.
7
290
Josh Morrison retweeted
Scott Aaronson four days ago on the possibility that AI models will soon break widely used cryptographic systems, and on what responsible disclosure looks like when the parties who need to be notified are effectively the entire world.
