Free AI crawler checker
Check both sides of access: what your robots policy says and what an AI-crawler request actually receives from your server or CDN.
A crawler can be allowed in robots.txt and still receive a firewall challenge, empty shell or error response.
See the exact rule matched for each crawler identity.
Separate an allowed policy from a successful content fetch.
Retrieval, search and model-training crawlers have different purposes.