Technical Briefing: Keeping OpenAI Crawlers Unblocked — IP Feeds, NGINX Rate-Limit Bypass, and fail2ban Exemptions The Problem OpenAI publishes three crawler IP feeds — GPTBot, ChatGPT-User, and OAI-SearchBot — as JSON, each with a creationTime field and a prefixes array of ipv4Prefix entries. These ranges live in Microsoft Azure space and rotate, so any hard-coded list goes stale quickly. The practical failure mode is silent: robots.txt alone is insufficient. Rate limiting (limitreq/limitconn)…