“Misalignment and Strategic Underperformance: An Analysis of Sandbagging and Exploration Hacking” by Julian Stastny, Buck Shlegeris

Subtitle: A new analysis of the risk of AIs intentionally performing poorly..

In the future, we will want to use powerful AIs on critical tasks such as doing AI safety and security R&D, dangerous capability evaluations, red-teaming safety protocols, or monitoring other powerful models. Since we care about models performing well on these tasks, we are worried about sandbagging: that if our models are misaligned1, they will intentionally underperform.

Sandbagging is crucially different from many other situations with misalignment risk, because it involves models purposefully doing poorly on a task, rather than purposefully doing well. When people talk about risks from overoptimizing reward functions (e.g. as described in What failure looks like), the concern is that the model gets better performance (according to some metric) than an aligned model would have. And when they talk about scheming, the concern is mostly that the model gets performance that is [...]

---

Outline:

(03:55) Sandbagging can cause a variety of problems

(08:13) Training makes sandbagging significantly harder for the AIs

(09:58) Training on high-quality data can remove sandbagging

(12:51) If off-policy data is low-quality, on-policy data might help

(15:12) Models might subvert training via exploration hacking

(19:31) Off-policy data could mitigate exploration hacking

(24:56) Quick takes on other countermeasures

(26:49) Conclusion and prognosis

---

First published:
May 8th, 2025

Source:
https://redwoodresearch.substack.com/p/misalignment-and-strategic-underperformance

---

Narrated by TYPE III AUDIO.