What Is the AI Alignment Problem? Yuval Noah Harari on When Machines Follow Orders Too Well

In Chapter 8 of Nexus, Yuval Noah Harari examines the AI alignment problem: how to ensure that artificial intelligence acts in ways that support human goals without causing harmful or unintended consequences.

The central risk isn’t necessarily that AI will rebel. A system can follow its instructions accurately and still produce disastrous results because its objective is incomplete, poorly defined or detached from the context in which it operates.

Mark and Jeremy discuss Harari’s examples of obedience, incentives and unintended outcomes, from Stalin-era loyalty tests to modern recommendation algorithms.

In this episode, we discuss:

• What the AI alignment problem is

• Why obedient AI systems can still cause harm

• How Stalin’s applause test illustrates dangerous incentive structures

• What Napoleon’s victories reveal about intelligence and long-term judgement

• How the paperclip maximiser thought experiment explains misaligned objectives

• Why fixed rules such as Asimov’s Three Laws can’t resolve every ethical conflict

• How social media algorithms manipulate attention and emotion

• Why current recommendation systems already demonstrate alignment failures

• Whether AI safety can be reduced to rules, constraints or technical safeguards

The episode distinguishes between rogue AI and a more immediate problem: systems that pursue the goals humans give them without understanding the values, trade-offs and consequences behind those goals.

AI alignment isn’t only about stopping machines from disobeying us. It’s about deciding what we should ask them to do, how those objectives should be interpreted and who bears responsibility when the result causes harm.

Please enjoy the show.

--

Timestamps

[00:00] Introduction: Books That Change Minds

[01:04] Diving into Nexus Chapter 8

[01:37] The Stalin Test: When Applause Becomes Terror

[06:11] Evolution of AI Principles

[07:45] Understanding the Attention Economy

[08:45] How AI Targets Our Limbic System

[09:29] Inside Facebook: The Leaked Reports

[11:49] Napoleon's Warning for AI[

15:55] The AI Alignment Problem Explained

[17:49] Racing Against Time: Human Goals vs. Doomsday Clock

[20:04] The Power of Divergent Thinking

[21:50] Understanding Deontology in AI Ethics

[26:55] Can Mythology Guide AI?

[27:54] Exploring Inter-computer Realities

[33:50] Why Asimov's Laws Won't Save Us

[38:31] NPCs & The Future of Digital Consciousness