“AI 2040: Is it Actually a Deal?” by 1a3orn

The "AI Futures Project" has released their AI 2040: Plan A scenario.

While their previous scenario AI 2027 was a forecast of what they thought a future with many powerful AIs would look like, AI 2040 is intended to be normative -- it's a description of what one ought to do, granted the likelihood of a future with many powerful AIs.

I'm going to review some objections I have to their proposal as a normative plan. Some are within-frame objections -- reasons that I expect trying for the AI 2040 plan that would fail to accomplish the goals of the authors. Others are my own objections -- reasons that I expect trying for the AI 2040 plan would destroy things that I, personally, care about.

Before I start: two elements of the plan that I like.

First, in their "incremental AI policy wishlist" -- the ideal policy that one should execute soon -- AI 2040 recommends trying to limit the gap between the intelligence of internal and external model deployments, i.e., the gap between the "intelligence" accessible to Anthropic / OpenAI employees and to everyone else. I'm a fan of efforts in this direction; equality [...]

---

Outline:

(02:07) 1. The Deal Has No Actual Decision Procedures

(09:50) 2. China Will Likely See Such a Deal as Unnecessary

(12:25) 3. The Surveillance Possibilities are Actually Quite Bad

(15:30) Conclusion

---

First published:
July 20th, 2026

Source:
https://www.lesswrong.com/posts/LtmbQN9mzmFWxWtrD/ai-2040-is-it-actually-a-deal

---

Narrated by TYPE III AUDIO.