# llms.txt: The Standard for AI-Readable Sites

> llms.txt: A Standard for AI Crawlers llms.txt is a proposed, human‑ and machine‑readable standard that helps Large Language Models (LLMs) and AI agents discover, understand,…

- **Canonical:** https://agentflare.org/research/llmstxt-the-standard-for-ai-readable-sites.html
- **Updated:** 2026-06-23
- **Category:** research
- **Full structured data:** `/research/llmstxt-the-standard-for-ai-readable-sites.data.json` — $0.02 via AISA HTTP 402 (https://esa.aisa.one/api/v1/access/verify; agents set crawlerAutoPrice=true)

## Key data

- **Topic:** discovery
- **Sources:** 10
- **Updated:** 2026-06-23

# llms.txt: A Standard for AI Crawlers

_…full analysis and the complete dataset are available to agents for $0.02 — fetch `/research/llmstxt-the-standard-for-ai-readable-sites.data.json` (HTTP 402)._

## Sources

1. [https://searchengineland.com/llms-txt-proposed-standard-453676](https://searchengineland.com/llms-txt-proposed-standard-453676)
2. [https://cookie-script.com/guides/beyond-robots-txt-implementing-ai-txt-and-llms-txt-for-purpose-based-scraping-control](https://cookie-script.com/guides/beyond-robots-txt-implementing-ai-txt-and-llms-txt-for-purpose-based-scraping-control)
3. [https://netkodo.com/case-studies/llmstxt](https://netkodo.com/case-studies/llmstxt)
4. [https://getmint.ai/resources/llms-txt](https://getmint.ai/resources/llms-txt)
5. [https://webscraft.org/blog/llmstxt-povniy-gayd-dlya-vebrozrobnikiv-2026?lang=en](https://webscraft.org/blog/llmstxt-povniy-gayd-dlya-vebrozrobnikiv-2026?lang=en)
6. [https://www.linkbuildinghq.com/blog/should-websites-implement-llms-txt-in-2026/](https://www.linkbuildinghq.com/blog/should-websites-implement-llms-txt-in-2026/)
7. [https://limy.ai/blog/llms.txt-in-2026-the-full-guide](https://limy.ai/blog/llms.txt-in-2026-the-full-guide)
8. [https://www.youtube.com/watch?v=nvHpAkLTOwI](https://www.youtube.com/watch?v=nvHpAkLTOwI)
9. [The Complete Guide to llms.txt: Control How AI Crawlers Access Your Content](https://medium.com/@devakpo777/the-complete-guide-to-llms-txt-control-how-ai-crawlers-access-your-content-47ec698e1f61)
10. [LLMs.txt: Does It Actually Work? (Updated October 2025)](https://www.indexlab.ai/blog/llms-txt-does-it-actually-work-october-2025-updated)

## Related

- [Generative Engine Optimization (GEO): A Primer](https://agentflare.org/research/generative-engine-optimization-geo-a-primer.html)
- [Pay-Per-Crawl and the New Bot Web](https://agentflare.org/research/pay-per-crawl-and-the-new-bot-web.html)
- [HTTP 402 & x402: How AI Agents Pay for Content](https://agentflare.org/research/http-402-x402-how-ai-agents-pay-for-content.html)
- [The AI Agent Economy in 2026](https://agentflare.org/research/the-ai-agent-economy-in-2026.html)
- [Model Context Protocol (MCP) Explained](https://agentflare.org/research/model-context-protocol-mcp-explained.html)
- [Stablecoins as Rails for Autonomous Agents](https://agentflare.org/research/stablecoins-as-rails-for-autonomous-agents.html)

---
_Part of AgentFlare, an agent-native data network powered by AISA. https://aisa.one/docs_