AI News

Diffusion language models can generate tokens in parallel, but they typically require multiple sampling steps to produce high-quality text. This paper introduces Seq-Drifting, a framework that operates on sequences of continuous token representations and learns a direct mapping from Gaussian noise to a complete text sequence. Its training targets differ from those of existing diffusion language models: instead of supervising each position only with the observed token, Seq-Drifting adapts position-wise targets to the context currently being generated. Attraction pulls outputs toward contextually plausible token representations, while repulsion discourages sample collapse and repetition within a sequence. The resulting generator produces all tokens jointly in a single inference pass. On LM1B, it achieves a generative perplexity of 36.01 in one step and performs competitively in both unconditional and conditional generation. Additional experiments on translation, summarization, and reasoning indicate that the approach extends beyond open-ended generation. The paper presents sequence drifting as a possible route to effective one-step text generation and makes its code available on GitHub.