Getting Started with HTTP 402 Pay-Per-Crawl

How AI agents pay for web content using HTTP 402 and crawler tokens

AI agents can now pay for web content and APIs directly over HTTP using the revived 402 “Payment Required” status code and standards like x402 and AISA HTTP 402. This guide walks through the practical flow for both agent builders and publishers.

1. Publisher: Signal payment with HTTP 402 and crawler tokens

Publishers expose paid content or APIs by returning 402 Payment Required when a crawler or agent hits a protected endpoint. In the response, include:

• 402 Payment Required status
• Payment metadata headers (e.g., X-Payment-Amount, X-Payment-Asset: USDC, X-Payment-To, X-Payment-Network)
• Optional crawler token or agent identifier (e.g., X-Crawler-Token or User-Agent pattern) to rate‑limit or whitelist specific bots

For example, when an AI agent requests /api/news without a valid payment, the server responds with 402 and structured payment terms instead of blocking or redirecting to a human checkout.

2. Agent: Handle 402 and pay with crypto

When an AI agent receives a 402, it parses the payment descriptor and decides whether to pay. The agent:

• Uses a built‑in wallet (often USDC‑backed) to construct an on‑chain payment matching the 402 terms
• Sends the payment via a facilitator (e.g., Coinbase, Cloudflare Agents SDK, or AgentCore Payments)
• Retries the original request with a payment credential header (e.g., PAYMENT-SIGNATURE or PAYMENT-RESPONSE in x402, or an AISA‑compliant receipt header)

This flow is fully programmatic: no human approval, no sessions, and no traditional API keys for payment.

3. Publisher: Verify and serve content

On the retry request, the publisher or its facilitator:

• Verifies the on‑chain transaction (amount, asset, recipient, and window)
• Optionally checks the crawler token or agent identity against allow‑lists or rate‑limits
• Returns 200 OK with the requested content or data

AISA HTTP 402 and x402 both treat payment as a first‑class HTTP primitive, so the same pattern works for AI agents, crawlers, and human‑driven clients.

Key takeaways

• Use 402 Payment Required with structured headers to signal payment terms to AI agents and crawlers, instead of opaque blocks or redirects.
• AI agents pay autonomously via crypto (e.g., USDC on Base/Ethereum) using x402 or AISA HTTP 402, then retry with a signed payment receipt.
• Publishers verify on‑chain payments and crawler tokens server‑side, enabling pay‑per‑crawl without accounts, sessions, or human checkout.

Free guide synthesized by the AISA LLM layer (AISA Perplexity API). 2026-06-23.