Cloudflare Launches Pay per Crawl to Charge AI for Web Scraping

Cloudflare, which provides infrastructure for about 20% of the internet, is introducing a system called Pay per Crawl. This initiative would enable publishers to require payments from AI companies every time their bots scrape website content. The move follows a year of Cloudflare building tools to block unwanted bots and could reshape the economics of online information sharing. On TechCrunch’s Equity podcast, hosts Kirsten Korosec and Max Zeff discussed how Pay per Crawl is a natural extension of Cloudflare’s efforts to help publishers protect and monetize their work. Some view the plan as an innovative way to ensure AI companies fairly compensate content creators, while others question whether it will gain widespread adoption or simply remain an ambitious idea. The episode also highlighted other significant tech stories. ICEBlock, an app that allows people to anonymously report U.S. Immigration and Customs Enforcement sightings, surged in popularity after criticism from former prosecutor Pam Bondi, quickly becoming one of the most downloaded free iPhone apps in the country. Meanwhile, Figma filed its S-1 paperwork, signaling a major step toward a public offering. The design platform reported 48% revenue growth, demonstrating strong demand in the creative tools market. In another development, Grammarly announced its acquisition of Superhuman, indicating its ambitions to build advanced productivity tools using AI. The show also covered Tesla co-founder JB Straubel’s latest venture, which could challenge Tesla’s dominance in the energy storage sector. Equity, TechCrunch’s flagship podcast produced by Theresa Loconsolo, continues to explore key shifts in technology and business. The program airs every Wednesday and Friday. For U.S. listeners, the hosts wished everyone a relaxing holiday weekend.