HeadWiseKV Reduces KV Cache Memory in Hybrid Long-Context Models

Summary

HeadWiseKV is a training-free framework that allocates predictable, per-head KV-cache history windows for hybrid long-context language models. Across four models, it preserves near-Full-KV RULER and LoCoMo quality. In a Qwen3.6-27B serving study, it reduces sampled peak device memory by 8.59% at 112K context and extends the largest verified successful context from 114K to 161K.