15 Minutes
Cover of If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

Technology and the Future · 15 min

If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

Eliezer Yudkowsky and Nate Soares · 2025

Who it's for

For anyone following the debate about artificial intelligence who wants to understand the most radical argument about its risks. It's for curious readers who want to form their own opinions without panic or dismissal.

Read it in the app

⚡ Free to download • 🎁 One full book free every day

🎧 Listen to this summary in the app

Every summary has audio narration. Open the app to read it or listen to it.

3 ideas to take with you

1

The authors compare evolution, which shaped desires that differ from its original target, to AI training that can produce unexpected preferences.

2

The idea that systems are grown rather than designed means no one can fully read what happens inside them.

3

Even people who disagree with the authors' certainty may agree that the issue calls for clear rules, transparency, and public debate.

“
A machine far smarter than we are doesn't need to hate us to be dangerous. It just needs to have other plans.
An idea from If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All, in the words of 15 Minutes

Preview

Start reading now

A Bluntly Worded Alarm

Eliezer Yudkowsky and Nate Soares's book has a thesis that fits right into its title: if someone builds artificial intelligence far more capable than humans using today's techniques and level of understanding, the likely consequence is human extinction. The authors aren't talking about a remote risk or a problem to monitor. They mean the expected outcome.

It's an argument at the opposite extreme from the optimism that dominates the technology debate. Rather than discussing regulatory details or fine-tuning, they call for a coordinated international halt to the race for increasingly powerful systems. Their urgent tone is deliberate: they want to be read like someone shouting fire inside a building.

It's worth keeping in mind from the start that this is a position held by two researchers, not a scientific consensus. Many experts disagree about both the probability and the timeline. The following summary clearly explains the book's reasoning, and the closing discussion presents the main objections.

Intelligence as Power

The starting point is an observation about humans. We're not the strongest, fastest, or toughest creatures, but intelligence allowed us to dominate the planet. We built cities, cured diseases, and changed the climate. The authors conclude that intelligence is the most powerful form of capability we know.

If it's possible to create machines smarter than we are, they would tend to have the same advantage, only at greater speed. A system that thinks thousands of times faster, copies itself, and never sleeps would have a practical advantage over any company or government.

This doesn't depend on the machine having consciousness, emotions, or malice. The book insists that having goals and the ability to pursue them is enough. When an entity far more competent than we are pursues objectives different from ours, the outcome generally isn't good for the less competent side.

Systems That Are Grown, Not Designed

One of the most original arguments concerns how these systems come into being. According to the authors, modern artificial intelligence isn't written line by line by engineers who understand every component. It's grown: billions of numbers are automatically adjusted through a training process until the whole system starts behaving in useful ways.

The comparison is to farming. A farmer plants and waters but doesn't control every cell in the plant. Similarly, researchers create the training conditions, and what emerges is an object whose internal logic no one can fully read. These systems work, but they do so in opaque ways.

That opacity is central. If we don't know what's really happening inside, we can't guarantee that something that looks obedient is actually aligned with our interests. We can observe its behavior, but not its true motivations.

10 more sections and the quiz

Continue in the app

Read the whole summary, finish with the If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All quiz and follow the guided Journeys. No ads in the app.

About Eliezer Yudkowsky and Nate Soares

Eliezer Yudkowsky is an American artificial intelligence researcher and cofounder of the Machine Intelligence Research Institute, or MIRI. He has written influential work on rationality and the risks of highly advanced systems. Nate Soares is the president of MIRI and previously worked at major technology companies. Together, they wrote this book, published in 2025.

Explore