Jürgen Schmidhuber - Part 2: JEPA, the Road to AGI, and Who Really Invented Modern AI

In the second half of our conversation with Jürgen Schmidhuber, we focus on the key ideas he's pursued since the early 1990s and discuss why he believes these concepts are only now being rediscovered.

We start with JEPA. Jürgen argues that the method LeCun named in 2022 is the same family he published in 1992 as Predictability Maximization. From there he traces the adversarial lineage back further still, to his 1990 world-model paper and 1991 Predictability Minimization  -  the curiosity-driven minimax games he sees as the real origins of GANs.

We also talk about why these ideas took thirty years to land, why today's trillion-dollar data-center buildout is driven by AGI fear, and why he thinks Apple may come out ahead.

The back half turns to what he sees as the real frontier: physical AI. Today's systems are superhuman behind the screen but helpless at a leaky pipe, and until a robot can use human tools, there's no AGI. He discusses self-replicating, self-improving machines as "a new kind of life," reframes continual learning and test-time training as ideas from his 1991 fast-weight work, and detours through Solomonoff's universal prior, Hutter's AIXI, and the Gödel machine.

We close on the subject Jürgen is famous for: scientific credit. He makes his case for rigorous attribution, casts himself as a "speaker for the dead" championing forgotten pioneers like Ivakhnenko, and reflects candidly on whether the fights are personal.

Timeline

00:30 — What JEPA is, and the 1992 Predictability Maximization story

04:54 — Implementing PMAX: autoencoders, Siamese networks, Infomax

09:10 — Predictability Minimization, factorial codes, and the roots of GANs

16:00 — Why it took 30 years: the economics of compute

20:52 — Data, the web, and 1990 as the origin point

23:09 — Hardware inflation, the trillion-dollar buildout, and the coming crash

34:05 — Physical AI: the plumber problem and self-replicating machines

41:14 — Which 90s ideas are being scaled right now

45:26 — Continual learning and test-time training as "old hats"

55:19 — Measuring intelligence: Solomonoff, AIXI, and the Gödel machine

1:05:26 — Self-replication and von Neumann

1:09:51 — Will he see AGI in his lifetime?

1:10:42 — Credit, integrity, and being a "speaker for the dead"

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
• "Palms Down" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
• Changes: trimmed

• ---

About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.