Start / LessWrong (30+ Karma) / Claude 4 you safety and alignment by zvi

“Claude 4 You: Safety and Alignment” by Zvi

119 min • 25 maj 2025

Unlike everyone else, Anthropic actually Does (Some of) the Research. That means they report all the insane behaviors you can potentially get their models to do, what causes those behaviors, how they addressed this and what we can learn. It is a treasure trove. And then they react reasonably, in this case imposing their ASL-3 safeguards on Opus 4. That's right, Opus. We are so back.

Yes, there are some rather troubling behaviors that Opus can do if given the proper provocations. If you tell it to ‘take initiative,’ hook it up to various tools, and then tell it to fabricate the data for a pharmaceutical study or build a bioweapon or what not, or fooling Opus into thinking that's what you are doing, it might alert the authorities or try to cut off your access. And That's Terrible, completely not intended behavior, we agree it shouldn’t do [...]

---

Outline:

(03:44) Introducing Claude 4 Opus and Claude 4 Sonnet

(06:08) Activate Safety Level Three

(08:17) The Spirit of the RSP

(09:46) An Abundance of Caution

(10:34) Okay What Are These ASL-3 Precautions

(16:55) How Annoying Will This ASL-3 Business Be In Practice?

(18:34) Overview Of The Safety Testing Process

(20:12) False Negatives On Single-Turn Requests

(20:39) False Positives on Single-Turn Requests

(21:33) Ambiguous Requests and Multi-Turn Testing.