CF-Incidents AI Crawl Control - AI Crawl Control

[ENF News Bot]

Every site on Cloudflare now has access to AI Audit, which summarizes the crawling behavior of popular and known AI services.

You can use this data to:

• Understand how and how often crawlers access your site (and which content is the most popular).
• Block specific AI bots accessing your site.
• Use Cloudflare to enforce your robots.txt policy via an automatic WAF rule.

To get started, explore AI audit.

Continue reading...