Sveriges mest populära poddar
Redwood Research Blog
Redwood Research Blog

″[Paper] Distillation for Incrimination and Distillation for Capabilities” by Sebastian Prasanna, Alek Westover

21 min•9 oktober 2026

Om avsnittet

Subtitle: Empirical work on the Distillation Double Bind.

TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a paper empirically testing those ideas. Distillation for Incrimination (DFI) distills an untrusted model into a weaker, trusted one, aiming to transfer the teacher's misalignment but not its ability to hide it, so that auditing the student can incriminate the teacher. We test DFI on AuditBench's secret-keeping model organisms, each fine-tuned to have a hidden quirk and trained to deny it. Distilling them back into their underlying instruction-tuned model yields students that admit the quirk far more often than the organisms do, suggesting that the quirk transferred faster than the propensity to hide it. We find that DFI doesn’t work well when the student does not share the teacher's pretrained base, so DFI should target the teacher's own pre-RL checkpoint, which is weaker than the teacher but shares its base model. Distillation for Capabilities (DFC) aims to transfer capabilities but not misalignment, resulting in a stronger trusted model. We identify promising DFC methods, showing that training for more epochs on fewer unique examples (holding total training steps fixed), and inoculation prompting, can preserve the capability [...]

---

Outline:

(01:59) Recap of the Distillation Double Bind

(05:41) Distillation for Incrimination

(05:45) Setup

(07:38) Results

(10:46) Failure modes of DFI

(12:59) Distillation for Capabilities

(13:03) Setup

(15:06) Results

(18:07) Conclusion

The original text contained 7 footnotes which were omitted from this narration.

---

First published:
October 9th, 2026

Source:
https://blog.redwoodresearch.org/p/paper-distillation-for-incrimination

---

Narrated by TYPE III AUDIO.

---

Images from the article:




Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Redwood Research Blog med Redwood Research finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.