Cloudflare Takes Aim at AI Scraping with Pay-Per-Crawl Plans

Cloudflare is taking on AI scraping by offering pay-per-crawl and pay-per-use plans to help publishers make licensing deals. The company's chief strategy officer, Stephanie Cohen, says their blocks are effective in preventing bots from getting through. However, some critics argue that the blocks may not be foolproof, as there are many ways for bots to bypass them.

Cohen suggests that if publishers are being scraped, it may be due to leaky buckets elsewhere on their website, such as syndicated content that allows bots to access it. This is a concern for health charities, which have seen a decline in website traffic due to Google AI Overviews and 'zero-click' searches.

Despite this, people are still seeking out charity websites with a clear purpose in mind, and donations continue to come through. The situation highlights the challenges faced by publishers in the age of AI and the importance of effective blocking measures.