IdeaLens Detects Whether Ideas in Long-form Writing Came from Humans or AI

Summary

IdeaLens is a detector designed to answer a question that conventional prose detectors do not: whether a document's ideas originated with a human or an AI, regardless of who wrote the final wording. The researchers convert each document into an outline whose items pair discourse roles with brief paraphrases, reducing word-level overlap with the source text. They train the system on 1 million FineWeb documents using silver labels generated by Pangram, a prose provenance detector, so that the remaining signal is expected to lie mainly in the ideas. In a controlled study, IdeaLens's AI flag rate fell from 95% to 7% when models wrote from increasingly detailed human plans, while Pangram 4 still flagged 92% of the resulting text. When the plans themselves came from AI, IdeaLens continued to flag more than 96% of documents. On a separate dataset of 50 stories written by humans from AI-generated plans, IdeaLens flagged 68%, compared with 8% for Pangram 4. Across 19 existing detection benchmarks, the authors report strong detection at low false-positive rates, with performance maintained across domains, formats, and languages. They also analyze 90,000 IdeaLens predictions to study systematic differences between human and AI ideation. The researchers release the models and labeled datasets for further research, but the supplied abstract does not establish how the detector performs outside the reported benchmarks or how its labels should be used in policy decisions.