When Good Verifiers Go Bad: Self-Improving VLMs Can Regress on New Tasks

Self-improving AI — where a model uses a verifier to generate its own training feedback — sounds like a path to perpetual improvement, but this paper shows it can silently make models worse. The key problem is task specificity: a verifier that accurately scores math problems may perform near-randomly on multi-disciplinary reasoning, and when it does, it feeds the learner confidently wrong preference signals that degrade performance. Alarmingly, more accurate-but-still-wrong verifiers cause more damage than near-random ones. The takeaway is operational: teams deploying self-improvement loops must first validate verifier quality on the target task specifically, not just overall benchmark performance. This matters for any production ML team using RLHF-style pipelines. Authors: Jianzhe Lin Paper: https://arxiv.org/abs/2606.14629v1