Siliconangle2026-10-31 02:00:23

Google, Anthropic and others to support new AI crawling opt-out standard

Google LLC, Microsoft Corp., Anthropic PBC and OpenAI Group PBC today announced plans to support an upcoming website crawling standard.

Summary

The technology will enable website operators to block artificial intelligence developers from using their content to train their models.

robots exclusion protocolgooglemicrosoftrobots.txtnewsaianthropicopenai

This is a brief wire summary, the full story (linked below) has the complete details.

KazaSec's take

Web and API vulnerabilities like this are exactly what manual penetration testing is designed to catch, automated scanners routinely miss the logic flaws and chained issues that cause the most damage in practice.

Coverage details

Source: Siliconangle
Published: 2026-10-31 02:00:23
Author: maria deutscher
Category: technology

We've archived 1318 other articles touching the same topic (robots exclusion protocol, google, microsoft) , see the full security news archive.

Worried this kind of story could be about you next?

We help organizations find and fix the gaps before they make headlines.

Request a Security Assessment Browse Services