Skip to content

The site does not say how AI may use its content

content-signals-missinginfo
On this page

Why it matters

  • robots.txt keeps a crawler from a page or lets it in, but does not say what it may do with the page once read. Content signals do: whether it may show in search, be input to AI answers, and be used to train models.
  • This is information: signals are an emerging convention some services follow and others do not, and their absence neither keeps anyone out nor lets anyone in.

How to fix

Add a line to the User-agent: * group with your preference:

text
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /

How we detect

In the agent-readiness tool alone: we read the Content-Signal and Content-Usage lines in robots.txt's groups, and the content-signal header of the page's response. One line with a defined key, or the header, is enough.