Why it matters
- robots.txt keeps a crawler from a page or lets it in, but does not say what it may do with the page once read. Content signals do: whether it may show in search, be input to AI answers, and be used to train models.
- This is information: signals are an emerging convention some services follow and others do not, and their absence neither keeps anyone out nor lets anyone in.
How to fix
Add a line to the User-agent: * group with your preference:
text
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /How we detect
In the agent-readiness tool alone: we read the Content-Signal and Content-Usage lines in robots.txt's groups, and the content-signal header of the page's response. One line with a defined key, or the header, is enough.