Sveriges mest populära poddar
Generally Intelligent
Teknologi

There will be a scientific theory of deep learning

1 tim 33 min•24 april 2026

Deep learning works extraordinarily well. And we still largely don't know why.

A new paper from Jamie Simon, Daniel Kunin, and 12 co-authors argues that a scientific theory of deep learning is emerging, and coins a name for the emerging field: learning mechanics.

We sat down with Jamie and Dan on Generally Intelligent to talk about what a physics of deep learning would actually look like, why now, and what's left to figure out.

00:03:05 Learning mechanics as the physics to mechanistic interpretability's biology

00:04:13 Why deep learning needs a theory

00:07:07 Why deep learning is uniquely hard to engineer

00:12:11 How a week in the woods became a paper

00:25:59 The barrier to theory isn't opacity, but complexity

00:36:26 Deep learning's first gas law

00:47:22 Why more particles makes the problem easier

00:56:22 The discretization hypothesis

01:01:50 The strongest signal that a compact theory exists

01:05:07 The Platonic Representation Hypothesis

01:15:41 Why learning mechanics and mech interp need each other

01:25:29 Theory as safety infrastructure

Read the paper

Transcript and links

Learning Mechanics website

Full transcript: https://imbueai.substack.com/p/geoffrey-litt


Generally Intelligent is a podcast by ⁠⁠Imbue⁠⁠, a research company building toward a future where AI agents are open and accountable to their users, so people have more power in the digital world.

Fler avsnitt av Generally Intelligent

Visa alla avsnitt av Generally Intelligent
Malleable software and human agency with Geoffrey Litt
•1 tim 32 min
From lawless spaces to true liberty: rethinking AI's role in society
•1 tim 39 min
Rylan Schaeffer, Stanford: Investigating emergent abilities and challenging dominant research ideas
•1 tim 3 min
Ari Morcos, DatologyAI: Leveraging data to democratize model training
•1 tim 34 min
Percy Liang, Stanford: The paradigm shift and societal effects of foundation models
•1 tim 2 min
Seth Lazar, Australian National University: Legitimate power, moral nuance, and the political philosophy of AI
•1 tim 56 min
Tri Dao, Stanford: FlashAttention and sparsity, quantization, and efficient inference
•1 tim 20 min
Jamie Simon, UC Berkeley: Theoretical principles for how neural networks learn and generalize
•1 tim 2 min
Bill Thompson, UC Berkeley: How cultural evolution shapes knowledge acquisition
•1 tim 15 min
Ben Eysenbach, CMU: Designing simpler and more principled RL algorithms
•1 tim 46 min

Generally Intelligent med Kanjun Qiu finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.