Generative AI's Effects on Student Learning Tested in Randomized Experiment

Summary

This study examines how generative AI affects student learning in a randomized experiment. In proctored, in-person sessions, undergraduate students learned an unfamiliar topic and wrote an analytical essay either with access to off-the-shelf generative AI or without it. They then completed unaided knowledge tests and open-ended essays immediately and again one week later. AI access raised immediate test scores by 0.27 standard deviations, and the gain persisted after one week. Essay quality changed little while AI was available, but unaided essays written one week later showed improvements in style and relevance. Those delayed gains were larger for augmentation users, who used AI to explain concepts, while short-term gains among automation users, who used AI to generate text, disappeared once AI access ended. The authors identify two possible mechanisms: AI users shifted time from drafting toward reading and information search, and they reported greater enjoyment of learning.