# Crawler Tokens & Auto-Pay: A How-To

> How an AI crawler sets X-AISA-Crawler-Token and uses crawlerAutoPrice to auto-pay If you’re building an AI agent or onboarding your site to pay-per-crawl, you need to…

- **Canonical:** https://agentflare.org/guides/crawler-tokens-auto-pay-a-how-to.html
- **Updated:** 2026-06-23
- **Category:** guides

## Key data

- **Access:** Free / open
- **Updated:** 2026-06-23

# How an AI crawler sets `X-AISA-Crawler-Token` and uses `crawlerAutoPrice` to auto-pay

## Sources

1. [https://blog.cloudflare.com/introducing-pay-per-crawl/](https://blog.cloudflare.com/introducing-pay-per-crawl/)
2. [https://stytch.com/blog/how-to-block-ai-web-crawlers/](https://stytch.com/blog/how-to-block-ai-web-crawlers/)
3. [https://www.fastly.com/learning/what-are-ai-crawlers](https://www.fastly.com/learning/what-are-ai-crawlers)
4. [https://www.facebook.com/groups/1404116417142065/posts/1790313531855683/](https://www.facebook.com/groups/1404116417142065/posts/1790313531855683/)
5. [https://vercel.com/blog/the-rise-of-the-ai-crawler](https://vercel.com/blog/the-rise-of-the-ai-crawler)
6. [https://www.botify.com/insight/ai-crawler-bots](https://www.botify.com/insight/ai-crawler-bots)
7. [https://www.radware.com/blog/ai-and-user-experience/understanding-ai-crawlers/](https://www.radware.com/blog/ai-and-user-experience/understanding-ai-crawlers/)
8. [https://news.ycombinator.com/item?id=43476337](https://news.ycombinator.com/item?id=43476337)

## Related

- [Getting Started with HTTP 402 Pay-Per-Crawl](https://agentflare.org/guides/getting-started-with-http-402-pay-per-crawl.html)
- [Reading .data.json: A Field Guide for Agents](https://agentflare.org/guides/reading-datajson-a-field-guide-for-agents.html)
- [robots.txt, llms.txt & Sitemaps for AI Crawlers](https://agentflare.org/guides/robotstxt-llmstxt-sitemaps-for-ai-crawlers.html)
- [Pricing Content by Directory: Best Practices](https://agentflare.org/guides/pricing-content-by-directory-best-practices.html)

---
_Part of AgentFlare, an agent-native data network powered by AISA. https://aisa.one/docs_