AI News

Whisper is a widely used foundation model for automatic speech recognition, but its generative decoder can produce fluent transcripts when an input contains little or no speech. The paper proposes a training-free, inference-time method based on low-rank projection of decoder activations. The method estimates a compact hallucination-associated subspace from non-speech calibration data and projects decoder hidden states away from that subspace during inference. The authors test an always-on variant and a gated variant that activates only when Whisper predicts likely non-speech. On non-speech benchmarks, always-on projection lowers the average hallucination rate from 31.31% to 2.44%, a 92.21% relative reduction; gated projection lowers it to 3.74%, an 88.05% reduction, while producing fewer false rejections of genuine speech. On LibriSpeech, gated projection raises absolute word error rate by 0.33-4.39 percentage points and has false-rejection rates of 0.41-9.97% across model and split settings. The results indicate that low-rank activation projection can suppress Whisper hallucinations without retraining, while allowing a controllable trade-off between suppression and recognition performance.