Why it matters
- Some AI agents ask for a page as Markdown, to read its text and headings without markup and scripts, with far less work and fewer mistakes.
- A site that negotiates content (RFC 9110) answers that request with
text/markdown, and answers a browser with HTML as usual. - This is information: an emerging convention, which some services in front of sites offer ready-made.
How to fix
Answer Accept: text/markdown with a Markdown version of the page, and tell caches the answer varies with the header:
http
HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Vary: AcceptHTML stays the answer to any other request.
How we detect
In the agent-readiness tool alone: we ask for the page itself once with Accept: text/markdown and Cache-Control: no-cache, where robots.txt allows, and look at the answer's type alone. Another rule compares the Markdown's text with the page's Arabic.