SubQ 1.1 Small Targets 12M-Token Reasoning with Sparse Attention

Summary

Subquadratic has released the model card for SubQ 1.1 Small, its second Subquadratic Sparse Attention model. The company reports near-perfect needle-in-a-haystack retrieval from 1M to 12M tokens, a 99.12% RULER score at 128K tokens, and 64.5 times less compute than dense attention at a 1M-token context. The model was developed through staged context extension and roughly one trillion tokens of continued pretraining on naturally long artifacts. Initial deployments with select design partners are starting, with a broader lineup planned by the end of the year.