“How can we solve diffuse threats like research sabotage with AI control?” by Vivek Hebbar

Subtitle: Preventing research sabotage will require techniques very different from the original control paper..

Misaligned AIs might engage in research sabotage: making safety research go poorly by doing things like withholding their best ideas or putting subtle bugs in experiments. To mitigate this risk with AI control, we need very different techniques than those in the original control paper or Ctrl-Z, because research sabotage differs from those settings in two crucial ways:

• Firstly, the AI can’t cause a catastrophe by behaving badly on just one occasion. To derail the entire project of solving alignment, the AI probably has to take a large number of malign actions. This contrasts with some other threat models (like rogue internal deployments), where the AI might succeed by acting badly on just a few occasions.

• Secondly and relatedly, individual bad actions in research sabotage aren’t very incriminating. If a model [...]

---

Outline:

(03:12) Threat model and basic countermeasures

(03:55) Deployer strategies: Training and incrimination

(08:34) Sabotage strategies

(08:56) Targeting sabotage at the most effective points

(09:33) Withholding good content vs. producing bad content

(11:52) Code vs. ideas

(12:29) What terms should we use for the concentrated/diffuse spectrum?

(14:58) Canary string

(15:12) Acknowledgements

---

First published:
April 30th, 2025

Source:
https://redwoodresearch.substack.com/p/how-can-we-solve-diffuse-threats

---

Narrated by TYPE III AUDIO.

---