Ask HN: Is AI Bot Traffic Out of Control?

Summary

The operator of a web app with about 50,000 statically generated public pages says bot traffic has reached an unprecedented level. Facebook’s crawler alone requested the same pages more than 3 million times over two days. The operator says attempts to block traffic through Vercel and Cloudflare have not solved the problem, while AI scrapers are routing requests through residential proxies. Even placing the entire app behind an authentication gate with Cloudflare Turnstile did not stop them: the scrapers allegedly created accounts to access data intended for people to read and use. The resulting traffic has caused hosting costs to rise sharply and has undermined the operator’s confidence in the reliability of online information. As a further countermeasure, the operator plans to try hCaptcha, which they believe may be harder for the bots to overcome. The post asks whether other website operators are seeing similarly sophisticated bot activity, but it does not provide independent traffic measurements or a confirmed explanation for every request.