Content Signals (search, ai-input, ai-train)
Also known as: Content-Signal, Cloudflare Content Signals Policy
A line in robots.txt that separates three different permissions people used to grant by accident all at once: indexing, answering, and training.
Last revised 2026-09-07. Also available as markdown — request this URL with Accept: text/markdown.
In plain English
"Allowed in robots.txt" used to mean one thing. Now a crawler might index you for search, quote you in an answer, or train a model on you — three uses with very different value to you, and until recently no way to distinguish them.
Content signals let you say which you accept: search=yes, ai-input=yes, ai-train=no allows indexing and citation while declining training. For most publishers that combination is the interesting one.
How to fix / set it up
- Decide the three permissions separately and write them down as a policy, not as a robots edit.
- Add the
Content-Signal:line to every user-agent group in robots.txt, not only to*. - Set
Google-ExtendedandApplebot-Extendedto match that decision. - Review it when your business model changes — the answer for a SaaS site and for a paid archive is not the same.
The technical detail
The line sits inside a robots.txt user-agent group, alongside Allow/Disallow: Content-Signal: search=yes, ai-input=yes, ai-train=no. It expresses a preference; it is not an enforcement mechanism, exactly like the rest of robots.txt.
Its weight is partly legal rather than technical: an explicit, machine-readable statement of terms is a stronger position than silence if use is ever disputed. Vendor-specific opt-outs such as Google-Extended and Applebot-Extended do a narrower version of the same job and remain worth setting.
A GEO-oriented site usually wants ai-input=yes. Declining it means declining to be quoted, which for most businesses gives up the thing they were trying to win.
FAQ
Does anyone honour this?
Compliant crawlers increasingly read it, and Cloudflare emits it by default for sites it fronts. Non-compliant ones ignore it, as they ignore everything else in robots.txt.
Should I set ai-train=no?
It depends on what your content is for. Declining training costs little in visibility today; declining `ai-input` costs you citations directly.