Media Watch: Anthropic, OpenAI and Microsoft Make It Easier to for Publishers to Opt Out of Being Scraped
Right now, if publishers don’t want their sites to be scraped by AI companies, they have to block each company’s crawlers individually. But, notes Nieman Lab, on Oct. 30, Anthropic, OpenAI, Google, and Microsoft said they would support a common robots.txt signal that, per OpenAI, “makes it …
Read the whole story at Nieman Lab.