Handling 402 and 403 Responses: An Error Reference for Agents

This guide is for developers building agents or crawlers that fetch paid publisher content through the AISA pay-per-crawl network. You will learn to read the JSON body of a 402 or 403 denial, branch on the error code, follow the guidance block, and avoid wasted retries. You will also know when to hand a payment link to the user and how to explain a blocked URL in plain language. A reference table of every denial code is included.

How the flow works

Before you start

• A registered AISA crawler account and its crawler token
• Basic HTTP and JSON knowledge
• curl or Python with the requests library

Step by step

1. Send your token and read the body

   Request the publisher URL with your crawler token and an honest User-Agent. The header is X-Crawler-Token; it replaces the earlier name X-AISA-Crawler-Token that older AISA pages still show. Never put the crawler token in Authorization: Bearer. A 200 means content, free or paid. A 402 or 403 carries a self-describing JSON body, and denials never charge your balance. Check the error field first.

   curl -s -i 'https://PUBLISHER-SITE/research/report.data.json' \
     -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN' \
     -A 'MyBot/1.0 (+https://example.com/bot)'

   S1 · esa.aisa.oneS2 · esa.aisa.one

2. Branch on the error code

   Parse the JSON and switch on error. Each code maps to one action: invalid_token means fix or register, insufficient_balance means the user must fund the account, the price states mean read the quote, and access_blocked means skip this URL. Always print guidance.steps as well, because the same system that generated them judges your retry. Do not retry blindly on any code.

   import requests
   
   url = 'https://PUBLISHER-SITE/research/report.data.json'
   headers = {'X-Crawler-Token': 'YOUR_CRAWLER_TOKEN',
              'User-Agent': 'MyBot/1.0 (+https://example.com/bot)'}
   resp = requests.get(url, headers=headers)
   if resp.status_code in (402, 403):
       body = resp.json()
       code = body.get('error')
       if code == 'invalid_token':
           print('Register or fix the header, then retry this URL')
       elif code == 'insufficient_balance':
           print('Shortfall:', body.get('crawler', {}).get('shortfall'))
       elif code == 'access_blocked':
           print('Skip this URL and read pricing.json')
       else:
           print('Price quote:', body.get('price'), body.get('currency'))
       for step in body.get('guidance', {}).get('steps', []):
           print(step)

   S2 · esa.aisa.one

3. Resolve 402 payment-side errors

   For invalid_token, the token is missing, unknown or inactive: register or correct the header and retry the same URL. A 402 body can carry guidance.registerApi; if it points at a different AISA host, prefer that host. For insufficient_balance, read crawler.shortfall, fetch the real balance, and create a Stripe checkout session (minimum 0.50 USD). The agent cannot pay; give the checkout_url to your user and wait.

   # Real balance with the crawler token
   curl -s 'https://cdn.aisa.one/api/agent/balance' \
     -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN'
   
   # Checkout link for the user (session token, not crawler token)
   curl -s -X POST 'https://cdn.aisa.one/api/agent/topup/create-session' \
     -H 'Content-Type: application/json' \
     -H 'Authorization: Bearer SESSION_TOKEN' \
     -d '{ "amount": 5.00 }'

   S1 · esa.aisa.oneS2 · esa.aisa.oneS3 · esa.aisa.one

4. Pivot on 403 access_blocked

   A 403 access_blocked is the publisher's policy choice, not a payment problem. Topping up or retrying will not unlock that URL. Do not stop the task: fetch the site's pricing.json, which lists purchasable pay-mode directories, and llms.txt, which describes content tiers. Both are free to read. Then continue with content that is for sale and explain the difference to the user.

   curl -s 'https://PUBLISHER-SITE/pricing.json'
   curl -s 'https://PUBLISHER-SITE/llms.txt'

   S2 · esa.aisa.oneS3 · esa.aisa.one

5. Follow guidance and report to the user

   Every denial includes a guidance block: onboarding, docs, llmsTxt, skill, registerApi, numbered steps, an optional signupBonus notice, and agentAdvice. Follow steps verbatim and narrate them. Tell the user whether content was free or paid, the exact amount charged, and the remaining balance, fetched fresh from the balance endpoint. For a 403, say plainly that the publisher withholds it from crawlers.

   402 insufficient balance: Your balance is short by the shortfall amount. Open this link to add funds: CHECKOUT_URL. I will continue after you pay.
   403 access blocked: The publisher does not allow crawlers on this URL, and payment cannot change that. I will skip it and fetch purchasable content from pricing.json instead.

   S2 · esa.aisa.oneS3 · esa.aisa.one

AISA denial codes and next actions

Common pitfalls

• Sending the crawler token in Authorization: Bearer makes the request unauthenticated and returns 402 invalid_token. Use only the X-Crawler-Token header.
• Do not top up or retry after 403 access_blocked. Payment does not unlock it; move to purchasable directories instead.
• Never call POST /api/agent/balance/add to fund yourself. On insufficient_balance, give the user the Stripe checkout_url and wait.
• Fetch the balance from GET /api/agent/balance before reporting it; remembered values go stale after spending. Never print the full token, only a prefix.

Key takeaways

• Read error first, then follow guidance.steps; denials never charge your balance.
• 402 means a payment-side fix, with the user paying; 403 access_blocked means skip the URL and pivot.
• Always tell the user what happened: free or paid, amount charged, real remaining balance.

FAQ

How this page is made. Written from 4 primary sources — the AISA agent and publisher documentation and AgentFlare's own files (pricing.json, robots.txt, llms.txt, live responses, .data.json field lists). Each step links the sources it relies on; steps naming a header, endpoint, error code or figure that the sources do not contain are removed. Refreshed weekly.