wpnewsDaily WordPress news

WordPress news in your inbox, every weekday

The day's top stories with every source. Free. One click to leave.

Topic · #54 in WordPress Pulse

robots.txt

Wider Tech22 Oct 2026

Legal experts say Meta's AI could still scrape news sites

The Verge

Meta −60 AI training −30 robots.txt −30 AI crawlers −30

Wider Tech22 Oct 2026

Google: robots.txt governs crawling, not AI training

Search Engine Journal

Google +10 robots.txt 0 AI crawlers 0 AI training −10 Gemini 0 SEO +10