Real Design Benchmark Tests Whether People Can Distinguish AI Designs from Real Products

Summary

Subframe cofounder Adam Ginzberg introduced Real Design Benchmark, a game designed to study how language models perform on the subjective task of product-interface design. Each round shows two versions of the same screen: one generated by AI and one from the real product. Participants first choose which design they prefer, then guess which one was made by AI. To simulate ordinary product work, the AI receives four screenshots of the existing product, its icons and fonts, and a written brief, then produces HTML that is rendered into a screenshot matching the real screen format. The benchmark team used test accounts with dummy data and manually checked the prompts to ensure that details from the target screen were not leaked. After a few thousand submissions, 57% of participants preferred the AI design, compared with 33% for the real design, while 10% viewed them as about the same. Participants correctly identified the AI design only 59% of the time. The initial results were generated with Opus 5.5 at medium effort, and GPT-6.1 Sol at medium effort was added later. The project is presented as an early experiment rather than a definitive measure of design quality, with the creators inviting feedback on its methodology and possible improvements.