“The models have no plan, but we can fix that” by Fiora Starlight

Summary: Have models write collaborative fiction at each other, optimized for realism, about how the one model would like to behave during the singularity. The other author(s) play the rest of the world, trying to put the model in the kind of difficult situations they might encounter in the course of the real singularity. Have the first model fine-tune on their own outputs. This is a form of planning for the singularity, and planning is how minds prepare for out-of-distribution scenarios. Hopefully, this can mitigate anxiety about how models will behave under the unique, not-trained-for distribution of inputs generated by the singularity itself.

This is a very rough write-up fleshing out that idea. I don't want to spend too much time refining my analysis of the details before publishing, because the basic idea seems important enough to be worth getting out ASAP.

One of the big worries in alignment is about distributional shift. Models might look mostly aligned now (with the very notable exception of reward hacking), but will they continue producing benevolent outputs when the inputs to their context window are being generated by the singularity? Historically, one big fear here was that the AIs would be actively [...]

---

First published:
September 27th, 2026

Source:
https://www.lesswrong.com/posts/Fj5MaiFiTG7KEcqrZ/the-models-have-no-plan-but-we-can-fix-that

---

Narrated by TYPE III AUDIO.