Our own visit statistics are always on. Google Analytics, which helps us understand how people use our public pages (not your signed-in workspace), is enabled only with your permission. See our Privacy policy.

EverySearch Long-running crawl API

POST

API health

24h health
Normal
24h average time
—
  • Healthy / normal
  • Available
  • Limited
  • Mostly down
Normal

Structured site extraction (long-running): takes from a dozen seconds to several minutes. Use it only when crawl cannot get the data

Price $0.69 per 1,000 requests ($0.000694 per call). Failed calls and empty results are not charged.

AI prompt

Paste this into your AI assistant. It already holds this endpoint's parameters, response fields and an example request, so the assistant can ask what you need and then write the scraper.

Help me write a script that calls the EveryInfra "Long-running crawl" API.

API contract:
- Request: POST https://api.everyinfra.com/api/v1/search with the header Authorization: Bearer <API Key>. Read the key from the EVERYINFRA_API_KEY environment variable; do not put it in the code.
- In the request body, set tool to harvest and put the parameters in params.
- Required parameters:
  - target: The URL of the page to extract from.
- Optional parameters:
  - fields: The field names to extract; each row returns values for these fields.
  - num: Maximum number of items to extract; it also caps the cost of the call.
- Example request body:
{
  "tool": "harvest",
  "params": {
    "target": "https://www.python.org/downloads/",
    "fields": [
      "release_version",
      "release_date"
    ],
    "num": 3
  }
}
- Billing: Price $0.69 per 1,000 requests ($0.000694 per call). Failed calls and empty results are not charged.
- On 422, fix the parameters using the suggestions and allowed values in the error. On 429 or 503, wait for Retry-After and try again.

Before writing code, ask me how many items I need to process and what format to save the results in, then confirm the plan and the rough cost with me.

Request parameters

NameInTypeRequiredDefaultDescription
AuthorizationheaderstringYes-Bearer followed by your API key. Create it in the console; never put it in a URL or a code repository.
toolbodystringYesharvestTool name, fixed value.
targetbody.paramsstringYes-The URL of the page to extract from.
fieldsbody.paramsarrayNo-The field names to extract; each row returns values for these fields.
numbody.paramsinteger (1–50)No10Maximum number of items to extract; it also caps the cost of the call.

Code samples

bash
curl -X POST 'https://api.everyinfra.com/api/v1/search' \
  -H "Authorization: Bearer $EVERYINFRA_API_KEY" \
  -H 'Content-Type: application/json' \
  --data-raw '{
  "tool": "harvest",
  "params": {
    "target": "https://www.python.org/downloads/",
    "fields": [
      "release_version",
      "release_date"
    ],
    "num": 3
  }
}'

OpenAPI definition (JSON)Samples read the key from the EVERYINFRA_API_KEY environment variable; copied code never contains your key.

Response example

Real response

Captured October 24, 2026 at 18:28 (UTC) by calling this API with the parameters in the code sample above. results and count are that call's real response with personal information (names, handles, avatars, profile links, contact details) removed; the list keeps the first 2. id, billing and quota differ on every call, so only their shape is shown.

json
{
  "id": "req_…",
  "tool": "harvest",
  "results": [
    {
      "release_version": "Python 3.15.1",
      "release_date": "Oct. 21, 2026"
    },
    {
      "release_version": "Python 3.15.0",
      "release_date": "Oct. 9, 2026"
    }
  ],
  "billing": {
    "charged": …,
    "charged_usd": …,
    "balance_after_usd": …,
    …
  },
  "quota": { … }
}

Errors and billing

401
Missing or invalid key.
402
Insufficient balance. Top up first.
422
Invalid parameter name or value. Fix it using the suggestions in the error. No charge.
429
Per-minute request limit reached. Wait for Retry-After, then try again.
503
Could not complete. The held charge is returned to your balance. If the outcome is unknown, verify it before you retry.
200 empty
No data. No charge.

Errors