Get a crawl
Pages by index. limit=0 returns the job only (for polling its progress).
GET
/v1/crawls/{id}Request
| Method and path | GET/v1/crawls/{id} |
|---|---|
| Base URL | https://api.boxline.dev |
| Authentication | Your API key in the x-api-key header (or Authorization: Bearer). The examples read it from BOXLINE_API_KEY. |
Path parameters
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
idrequired | string | – | – |
Query parameters
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
limit | integer | 50 | – | From 0 to 100. |
after | string | – | – | The next value of the previous page (same filters and sort). Opaque; do not build it yourself. |
curl "https://api.boxline.dev/v1/crawls/$CRAWL_ID?limit=50" \ -H "x-api-key: $BOXLINE_API_KEY"import { Boxline } from "@boxline/sdk";const bx = new Boxline(); // reads BOXLINE_API_KEY and BOXLINE_API_URLconst job = await bx.crawl.get("CRAWL_ID", { limit: 50 });console.log(job.status, job.pagesDone, job.data.length, job.next);// Every page of the crawl, following next:for await (const p of bx.crawl.pages("CRAWL_ID")) console.log(p.url);from boxline import Boxlinebx = Boxline() # reads BOXLINE_API_KEY and BOXLINE_API_URLjob = bx.crawl.get("CRAWL_ID", limit=50)print(job["status"], job["pagesDone"], len(job["data"]), job["next"])# Every page of the crawl, following next:for p in bx.crawl.pages("CRAWL_ID"): print(p["url"])Responses
200 OK
The job with data (pages) and next.
Fields
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
id | string | – | – | |
status | string | – | running, completed, failed, canceled | |
url | string | – | – | |
params | object | – | – | The crawl's settings (proxy without passwords). |
pagesDone | integer | – | – | |
pagesFailed | integer | – | – | |
skippedByRobots | integer | – | – | |
error | string | null | – | – | |
createdAt | string | – | – | Format: date-time. |
finishedAt | string | null | – | – | Format: date-time. |
data | object[] | – | – | |
data[].index | integer | – | – | |
data[].url | string | – | – | |
data[].finalUrl | string | null | – | – | |
data[].status | integer | null | – | – | |
data[].title | string | null | – | – | |
data[].depth | integer | – | – | |
data[].content | string | null | – | – | |
data[].captcha | string | null | – | recaptcha, hcaptcha, turnstile, cloudflare, datadome, arkose, human | |
data[].error | string | null | – | – | |
next | string | null | – | – | Pass it back as after for the next page; null on the last page. |
{ "id": "crawl_2Gh7Kx0Lq9Zp4Nm3", "status": "running", "url": "https://example.com/", "params": {}, "pagesDone": 0, "pagesFailed": 0, "skippedByRobots": 0, "error": "…", "createdAt": "2026-09-29T10:00:00.000Z", "finishedAt": "2026-09-29T10:00:00.000Z", "data": [ { "index": 0, "url": "https://example.com/", "finalUrl": "https://example.com/", "status": 0, "title": "Example Domain", "depth": 0, "content": "…", "captcha": "recaptcha", "error": "…" } ], "next": "eyJvIjoiY3JlYXRlZF9kZXNjIiwidCI6IjIwMjYtMDktMjlUMTA6MDA6MDAuMTIzNDU2WiIsImkiOiI2YzFmIn0"}Errors
Errors have a matching HTTP status and the same body. code is stable, so check it in your code; message is for people. See Errors and request ids.
| Status | Codes | When |
|---|---|---|
| 400 Bad Request | invalid_request, invalid_json, invalid_cursor, invalid_idempotency_key, invalid_extension, variables_with_extensions, terms_not_accepted, invalid_reset_token, invalid_confirmation_code, wrong_password, no_password, invalid_next, invalid_oauth_token | The request is not valid. |
| 401 Unauthorized | unauthorized | Missing or invalid API key. |
| 404 Not Found | not_found, invite_not_found, provider_unavailable | No such thing in this project (or organization), or no such route. |
{ "error": { "code": "not_found", "message": "session not found", "requestId": "req_4fJx0aQm2Lk9Pz7N" }}Learn more
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.