MIRI

A nuclear false alarm nearly ended the world 64 years ago today. How did we survive such incidents? What can it teach us about the current AI crisis? From IF ANYONE BUILDS IT, EVERYONE DIES: “There were close calls. During the height of the Cuban Missile Crisis, a U.S. ship dropped depth charges to try to force a Soviet submarine to emerge and identify itself. The submarine, B-59, thought a general war had started. B-59 was armed with a nuclear torpedo, use of which required the unanimous vote of three officers. Of the three, Vasily Arkhipov was the lone officer who dissented. But none of the close calls escalated to a full-scale nuclear war. One way or another, the people who thought that nuclear war would destroy everything in the coming decades ended up wrong. They were not wrong about the dangers. They weren’t wrong that a hydrogen bomb would flatten and burn a city, or about what it would be like to die of radiation poisoning, or about how an intercontinental rocket tipped with nuclear warheads would penetrate the best available defenses. Rather, they were wrong about humanity’s ability to decide not to die. We don’t know for sure why the war-prone peoples of Earth became that little bit wiser. But our guess is that, for the first time in human history, everyone who had the power to choose war expected to personally have a bad time if one broke out. Other wars might have put distant soldiers to ruin, but this one would bring ruin directly to their doorstep. The stronger power would also lose. Even so, it wasn’t simple or easy to avoid nuclear war. Negotiators worked tirelessly, for decade after decade, to avoid a single misstep that might lead to a fatal nuclear encounter. Countries had arms agreements and monitors. There was a direct line between U.S. leadership and Soviet leadership, in case somebody had to resolve a question very quickly. Humanity averted nuclear war because people who understood that the world was on track for destruction worked hard to change tracks. They did not see themselves as trying to prevent an improbable unlikely accident. They worked to un-write a fate already written. And civilization lived.”

"Anyone creating an ultra-intelligent machine anywhere is a threat to American national security, and so we need to somehow prevent that from happening[...] we need to start orienting to this the same way that we orient to preventing rogue actors from getting nukes."

"This is an extraordinarily important moment. If we’re able to convert the world’s newfound awareness into meaningful action, rather than insufficient half measures, we have a real chance." @robbensinger and Alana Horowitz Friedman on meeting the moment, MIRI updates, and more

.@ericschmidt asks an important question about an AI pause: "how do you verify it?" In a new post, technical governance researcher Naci Cankaya answers this question and more:

The Ban Artificial Superintelligence Act directly confronts the extinction threat that humanity is facing. We are inspired that the Sanders and Casar teams take this threat seriously, and MIRI endorses the Act. You can read more of our thoughts in the link below.

In celebration of still being alive and fighting, we are giving away 1000 digital copies of the NYT bestseller “If Anyone Builds It, Everyone Dies"

The AI situation has evolved since IF ANYONE BUILDS IT, EVERYONE DIES came out, a year ago today. But the global emergency described by the book still remains. We are sharing a few reflections on the book, the past year, and the present moment.

"I did a book club at Bridgewater, and said: everybody at Bridgewater's reading this book, so they understand the path that we're obviously on." On @TheStalwart's podcast, Bridgwater's CIO says he had all of the firm's employees read "If Anyone Builds It, Everyone Dies."

"If Anyone Builds It, Everyone Dies" includes an extinction scenario. Rather than being a prediction, it is meant to be one plausible example out of many ways that AI could go catastrophically wrong. Check out the video from AI in Context: youtu.be/Nl7-bRFSZBs

"Sollte irgendein Unternehmen oder eine Gruppe irgendwo auf der Welt eine künstliche Superintelligenz entwickeln, die auch nur annähernd auf den derzeitigen Techniken basiert und sich auch nur im Entferntesten auf das heutige Verständnis von KI stützt, dann werden alle Menschen auf der ganzen Welt sterben." Heute erscheint die deutsche Fassung von IF ANYONE BUILDS IT, EVERYONE DIES von @ESYudkowsky und @So8res im Buchhandel.

"The doomers are actually the optimists" MIRI CEO @m_bourgon talks to @heyseankim about why he has hope that humanity will rise to the challenge of preventing an extreme AI catastrophe.