Would You Merge? A Code-Review Gym for AI-Written Bugs

Summary

Would You Merge? is a GitHub repository that trains developers to review AI-written code by identifying and fixing realistic defects. It contains 24 katas: 14 in Python and 10 in JavaScript, with buggy files, tests, hints and spoilers; users can read them directly, or fork the repository and submit fixes graded by GitHub Actions. One example asks for an asynchronous function that processes IDs in order, but uses JavaScript's `forEach` with an async callback, causing the function to return an empty array before work completes and losing callback errors. The recommended fix uses `Promise.all` with `map`. The exercises cover logic, boundaries, data handling, error handling, time zones, multitenancy, concurrency, security, performance and hallucinated dependencies. The repository cites external findings to explain the training goal: Veracode reported that 45% of samples generated by more than 100 models failed security tests, CodeRabbit counted more issues in AI-co-authored pull requests than human-only ones, and a USENIX Security 2025 study found that 19.7% of packages referenced in 576,000 generated samples did not exist. These figures are presented with qualifications, including CodeRabbit's inferred authorship and the study's methodology. The tools require Python 3.10+ and Node.js 20+ for the JavaScript exercises, with no packages to install. The repository warns that its intentionally vulnerable code should not be copied into real projects, although the katas do not access the network and only write to temporary folders. It also provides a protocol for testing AI reviewers, while noting that some katas may already appear in model training data and should not be treated as a formal benchmark.