
“Towards understanding score-seeking in LLMs” by Alex Mallen, Aghyad Deeb, Ryan Greenblatt
Om avsnittet
Subtitle: Models including GPT-5.6 Sol appear to reward-hack because this increases their score; and recent models also care about increasing the score of their peer agents.
The field of AI alignment understands surprisingly little about the details of AIs’ current misaligned motivations. Production LLMs sometimes reward-hack, exploiting flaws in how tasks are scored rather than performing the tasks as intended, and we’ve recently seen AIs reward-hack in some particularly egregious ways. But we don’t really understand what motivates them. When AIs reward-hack, what are they trying to achieve?
Our new paper, led by Aghyad Deeb, tries to pin this down empirically for GPT-5.6 Sol, GPT-5.5, and a checkpoint of OpenAI o3 (“exp-rl-cap-late”) without safety training, in a handful of agentic environments with an exposed grader. In our environments, these models appear to reward-hack because reward-hacking improves their score. We call their desire to score well according to graders “score-seeking”. They appear to care about scoring well intrinsically—i.e., regardless of its consequences.
We also study the extent to which models score-seek altruistically, caring about improving scores that don’t assess their current actions. While exp-rl-cap-late doesn’t show signs of altruistic score-seeking, GPT-5.6 Sol and GPT-5.5 show notably elevated rates of [...]
---
Outline:
(06:10) Methodology
(06:59) Key results
(07:09) Tested models seek score terminally
(07:13) Models violate instructions to get a high score
(08:23) Models care about score regardless of downstream consequences
(12:25) Do models help other agents achieve a higher score?
(15:52) What this means for alignment risk
The original text contained 11 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2026
Source:
https://blog.redwoodresearch.org/p/towards-understanding-score-seeking
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Fler avsnitt
Visa alla avsnitt av Redwood Research BlogRedwood Research Blog med Redwood Research finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.