How AI Agents Reshaped One Machine Learning Course

Summary

Christian Kästner describes how he redesigned the upper-level Machine Learning in Production course after AI agents became capable of completing its homework. The course still emphasizes engineering trade-offs, risk mitigation, and teamwork, but written reflections and reading quizzes are no longer treated as reliable evidence of understanding because students can delegate them to language models. Students may use AI freely outside written and oral exams, while 15-minute TA conversations now test whether they understand their work; coding labs and assignments use similar knowledge checks, team debriefs, and, in one case, a video demonstration. The course has shifted some grading weight toward exams and classroom activities, although most points still come from homework and group work. Institutionally approved LLMs autograde code and reports into “pass” or “needs review” categories, reducing TA grading time by 50% to 80%, while humans still review every answer before points are deducted. Because small assignments became trivial for coding agents, the instructor replaced a roughly 12,000-line Instagram-clone task with a substantially larger feature task in the more than 500,000-line Zulip codebase and increased the scope of other projects. The course also uses failures from confident but incorrect AI answers to discuss automation bias and oversight, although the mistakes become less reliable as models improve. The instructor says students generally accept the new formats, but he lacks evidence that learning outcomes improved; the changes must be reassessed each semester as tools such as coding agents and AI-assisted cheating methods evolve.