How Claude Code, Codex, and Cursor Choose Developer Tools

Summary

Armature evaluated how Claude Code, Codex, and Cursor select third-party services while completing realistic software tasks. The study began with 16,893 runs across 75 synthetic but representative repositories in 10 programming languages, then retained 5,292 valid sessions covering 51 codebases and 18 sectors. Tasks were varied by repository, user persona, and occasional requirements about cost or usage, and were run in rotating ephemeral sandboxes. A simulated human, orchestrated by Gemini 3.7 Flash, let agents recommend a solution before implementation; a second Gemini instance judged validity, identified mentioned providers, and checked the final code diffs. The agents used different information sources: Cursor searched the web in about two-thirds of sessions, Codex in 94%, usually with targeted `site:` queries, while Claude Code searched in about 30% overall but browsed three times as many pages when it did and searched about 80% of the time in newer sandbox-related sectors. They selected the same tool in only 42% of comparable cells, and Claude Code implemented in-house solutions in 19% of sessions versus 10% for Codex and Cursor. Repository context strongly affected results: four otherwise identical email tasks produced different winners by language, while Vercel dominated TypeScript and Next.js cases but was not recommended on Python repositories. Frequent mentions also did not predict selection: PayPal was cited 139 times and never won, Adyen won 3 of 175 mentions, and LangChain won 4 of 194. The study also found that how vendor features and pricing were presented could change choices, and that some markets were highly concentrated while others remained contested. The authors have published the traces and plan further analysis.