AgentFlare

Guides · For publishers · Intermediate

How the Edge Verify API Decides Access

The edge gateway calls POST /api/v1/access/verify before fetching from origin and reads one field, accessGranted. Denials come back as 402 or 403 JSON that the gateway passes to the crawler unchanged.

LevelIntermediate
Reading time6 min
Steps52 with code
Sources4primary docs
✓ Open accessFree

You're reading the full report — free for humans. The machine-readable dataset for this section is open to all agents too — no payment required.

⚡ Agents: fetch .data.json → agents: allowed (no charge) · verify at https://esa.aisa.one/api/v1/access/verify

This guide is for publishers and CDN or ESA integrators who want to know exactly what an edge gateway sends to the AISA verify endpoint and what happens next. You will be able to build the verify request, read the decision, and return the right response to the crawler. You will also be able to tell free, paid and blocked outcomes apart by status code and error value, and to see why the 402 body must reach the crawler untouched. The guide ends with a table of every verify outcome.

How the flow works

For publishersDrawn from the AISA agent and publisher documentation
Publisher-side access decision flow: 1. Crawler to Your edge: GET /dir/file; 2. Your edge to AISA verify: POST verify: URL, token, UA; 3. AISA verify to Your edge: decision: allow / pay / block; 4. Your edge to Your origin: fetch when granted; 5. Your edge to Crawler: 200 content; 6. Your edge to Crawler: or 402 / 403 JSONCrawlerany agentYour edgeCDN / gatewayAISA verify/api/v1/access/verifyYour origincontent1. GET /dir/file2. POST verify: URL, token, UA3. decision: allow / pay / block4. fetch when granted5. 200 content6. or 402 / 403 JSON

Before you start

  • A gateway token issued by AISA out of band
  • Publisher rules configured as allow, pay or block
  • An edge gateway that can make server-side HTTPS calls

Step by step

  1. Send the verify call before origin fetch

    When a crawler request hits the edge, call POST /api/v1/access/verify before going to origin. Put the gateway token in Authorization: Bearer. Forward the crawler's own token in X-Crawler-Token, which replaces the earlier name X-AISA-Crawler-Token still shown on older AISA pages. Always forward the original crawler user agent, because gateways rewrite User-Agent. X-Request-Id is optional and is echoed back for reconciliation.

    http
    POST /api/v1/access/verify HTTP/1.1
    Host: cdn.aisa.one
    Authorization: Bearer YOUR_GATEWAY_TOKEN
    X-Crawler-Token: YOUR_CRAWLER_TOKEN
    X-ESA-Original-Crawler-Agent: ORIGINAL-CRAWLER-UA
    X-Request-Id: req-8c85ba19abef566b
    Content-Type: application/json
  2. Build the JSON request body

    Send publisherDomain (the request Host) and resourceUrl (the path; query strings are stripped before rule matching). Add crawlerAgent: the identified bot id, or the original user agent if unidentified. Add crawlerAutoPrice set to true so a paid resource is charged in one call when the balance allows. crawlerAutoPrice, crawlerExactPrice and crawlerMaxPrice are mutually exclusive; sending none means price discovery. Save these fields as verify-body.json.

    bash
    # verify-body.json holds: publisherDomain, resourceUrl,
    # crawlerAutoPrice (true) and crawlerAgent
    curl -X POST https://cdn.aisa.one/api/v1/access/verify \
      -H 'Authorization: Bearer YOUR_GATEWAY_TOKEN' \
      -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN' \
      -H 'Content-Type: application/json' \
      --data @verify-body.json
  3. Read accessGranted and decide

    A minimal gateway reads one field. When accessGranted is true, fetch the content from origin or cache and serve it with status 200. When it is false, do not serve content. Rules are first-match-wins: block gives 403 access_blocked, allow gives 200 free, pay needs a valid crawler token, and no matching rule gives 200 default allow with no charge. Charged responses carry charged and chargeAmount.

  4. Pass 402 and 403 bodies through unchanged

    On accessGranted false with statusCode 402 or 403, return statusCode and the verify JSON body to the crawler exactly as received. The body carries onboarding and a guidance block that teach a new crawler to register, fund a balance and retry with X-Crawler-Token. A 402 means payment can unlock the resource; a 403 access_blocked means payment will not help. Render your own block page if you want one, since AISA returns JSON only.

  5. Handle gateway-side failures separately

    A 401 gateway_unauthorized means the gateway token is missing or invalid, so check the key configured in the console; this is an operator error, not a crawler one. A 400 means a parameter problem such as invalid_price_format or conflicting price fields. A 500 internal_error is a provider fault, handled by your timeout and fault policy. On timeout retries, reuse the same X-Request-Id so both sides can reconcile charges.

Verify outcomes and gateway action

SituationStatus and erroraccessGrantedGateway returns to crawler
allow rule, or no rule matched (default allow)200, no chargetrueContent from origin, 200
pay rule, valid token, enough balance, crawlerAutoPrice true200 with charged and chargeAmounttrueContent from origin, 200
pay rule, token missing or invalid402 invalid_tokenfalse402 JSON passed through, with onboarding and guidance
pay rule, balance too low402 insufficient_balance, crawler.shortfallfalse402 JSON passed through, top up then retry
pay rule, no price intent (price discovery)402 price_required, price and currencyfalse402 JSON passed through with the price quote
exact or max price intent does not fit402 price_mismatch or price_too_highfalse402 JSON passed through, retry with the real price or a higher budget
block rule matched403 access_blockedfalse403 JSON passed through, payment cannot unlock
gateway or request errors401 gateway_unauthorized, 400 invalid_price_format, 500 internal_errorfalse401: check the gateway token in the console; 400: fix the request; 500: apply your fault policy

Common pitfalls

  • Never send the crawler token in Authorization: Bearer; that header is reserved for the gateway layer and yields 401 gateway_unauthorized.
  • Do not send crawlerAutoPrice, crawlerExactPrice and crawlerMaxPrice together; they are mutually exclusive and a conflict returns a 400.
  • Do not rewrite or trim the 402 JSON. Its onboarding and guidance fields are how new crawlers learn to register and pay.
  • The original-agent header is spelled X-ESA-Original-Crawler-Agent in AISA docs and X-Original-Crawler-Agent in the Open Verify spec; the body field crawlerAgent carries the same data.

Key takeaways

  • Read accessGranted: true means fetch from origin, false means return statusCode and the JSON body to the crawler.
  • 402 means payment can unlock the resource; 403 access_blocked means it cannot; 401 is reserved for the gateway layer.
  • Send the crawler token in X-Crawler-Token and the gateway token in Authorization: Bearer, never mixed.

FAQ

What does the gateway send to POST /api/v1/access/verify?

Authorization: Bearer with the gateway token, X-Crawler-Token forwarded from the crawler, the original crawler user agent header, and a JSON body with publisherDomain, resourceUrl, crawlerAutoPrice and crawlerAgent. X-Request-Id is optional.

What happens when the crawler has no token?

On a pay rule, verify returns 402 with error invalid_token, plus onboarding and guidance. The gateway passes that JSON to the crawler so it can register and retry. On an allow rule, no token is needed and the response is 200.

What is the difference between 402 and 403?

402 means the resource is for sale and payment or registration can unlock it. 403 with access_blocked means the publisher blocked it, regardless of token or balance, so the crawler should not register or top up for that URL.

How do I get price discovery without being charged?

Send none of crawlerAutoPrice, crawlerExactPrice or crawlerMaxPrice. For a pay resource, verify returns 402 with error price_required plus price and currency, and does not charge.

How this page is made. Written from 4 primary sources — the AISA agent and publisher documentation and AgentFlare's own files (pricing.json, robots.txt, llms.txt, live responses, .data.json field lists). Each step links the sources it relies on; steps naming a header, endpoint, error code or figure that the sources do not contain are removed. Refreshed weekly.

Sources & citations

  1. Header name: X-Crawler-Token esa.aisa.one
  2. Open Verify specification (edge gateway ↔ verify provider) esa.aisa.one
  3. AISA verify API for publishers and edge gateways (llms.txt) esa.aisa.one
  4. AISA publisher API documentation esa.aisa.one