Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders

Speech recognition has reached impressive accuracy on human speech, but what happens when a model confidently transcribes silence or background noise as coherent sentences? This hallucination problem in Whisper, a widely deployed transcription system, poses real dangers in medical dictation, legal transcription, accessibility tools, and automated meeting notes. This research demonstrates that the seeds of hallucination are detectable within the model's own internal representations, and that steering those representations can dramatically reduce false transcriptions. The approach requires no retraining, making it a practical intervention for anyone already deploying Whisper in production environments where reliability is non-negotiable. Authors: Georgii Aparin, Vadim Popov, Tasnima Sadekova, Assel Yermekova Paper: https://arxiv.org/abs/2606.07473v1