John Schulman on dead ends, scaling RL, and building research institutions

A conversation with John Schulman on the first year LLMs could have been useful, building research teams, and where RL goes from here.

00:00 - Speedrunning ChatGPT

09:22 - Archetypes of research managers

11:56 - Was OpenAI inspired by Bell Labs?

16:54 - The absence of value functions

18:23 - Continual learning

21:09 - Brittle generalization

24:05 - Co-training generators and verifiers, GANs

27:06 - John’s personal use of AI for research

28:54 - Day in the life

33:01 - Slowdowns in consequential ML ideas

36:21 - "Peer review" within the labs

39:19 - Distribution shift in researchers

43:33 - Future of RL

45:33 - Will the labs coordinate if the world needs them to?

44:46 - Forecasting ills in AGI and engineering

47:53 - Thinking Machines