The Citaim bot

Citaim fetches public web pages to run diagnostics — AI-readiness, security headers, SEO and DNS/email checks — either when someone asks us to check a site, or for our public readiness leaderboard. We aim to be a good citizen: we identify ourselves, respect robots.txt, and cryptographically sign our requests.

User-Agent

Citaim/1.0 (+https://citaim.com/bot; diagnostic scan)

Purpose

On-demand and benchmark diagnostics of public pages. We read server-rendered HTML only — no JavaScript execution, no logins, no form submissions.

Respects robots.txt

Yes. Our automated scanner honours Disallow rules for the Citaim token and the * group, and won’t crawl a site that disallows us.

Signed requests

Every request is signed with Web Bot Auth (HTTP Message Signatures, RFC 9421). Verify against our public key directory: /.well-known/http-message-signatures-directory.

Rate

Light and bounded — a small number of requests per host, with timeouts. We do not hammer origins.

JavaScript / cookies

None. We don't run JS, store cookies, or attempt to bypass anti-bot challenges — if a site blocks us, we back off.

Allow or block Citaim

To block us, add this to your robots.txt — we’ll comply:

User-agent: Citaim Disallow: /

To allow us through a WAF or bot-management rule, match our User-Agent token Citaim, or verify our Web Bot Auth signature. On Cloudflare, a WAF rule that skips bot protection when http.user_agent contains “Citaim” works; the cryptographic signature above is the tamper-proof way to confirm it’s really us.

Contact

Questions, or want us to stop crawling your site? Email [email protected] and we’ll act promptly. Citaim is operated by Northstar Infinity Works Ltd.