Fetch a page
Renders the page in a real browser inside a sandbox. 120 per minute per project (shared with screenshot and pdf).
POST
/v1/fetchRequest
| Method and path | POST/v1/fetch |
|---|---|
| Base URL | https://api.boxline.dev |
| Authentication | Your API key in the x-api-key header (or Authorization: Bearer). The examples read it from BOXLINE_API_KEY. |
| Rate limit | Yes. See Rate limits for the numbers and headers. |
Body
JSON with content-type: application/json. Required.
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
urlrequired | string | – | – | An http(s) URL. |
proxy | boolean | null | object | object[] | – | – | true = a residential US proxy; an object = one proxy; a list = rules by site (at most 10); null or false = none. |
proxy.typerequired | string | – | residential, datacenter, custom | |
proxy.country | string | – | – | Two-letter country code (residential and datacenter). |
proxy.state | string | – | – | Residential, US states, e.g. us_california (implies US). |
proxy.city | string | – | – | Residential, e.g. los_angeles (needs country). |
proxy.ip | string | – | sticky, rotating | Default sticky for sessions and crawls, rotating for fetch, screenshot, pdf and extract. |
proxy.scope | string | browser | browser, all | all also routes the session's shell. |
proxy.server | string | – | – | custom: your proxy, http://host:port or https://host:port, on a public address. |
proxy.username | string | – | – | |
proxy.password | string | – | – | Stored encrypted; never returned. |
proxy[].domainPattern | string | – | – | Up to 256 characters. |
browser | boolean | object | – | – | true or omitted: a browser with the defaults; false: none; an object: a browser with these settings. |
browser.mode | string | standard | standard, realistic | realistic puts the clock and language on the proxy's country. It never changes what the browser reports about itself. |
browser.locale | string | – | – | BCP 47, e.g. de-DE. Wins over the mode. |
browser.timezone | string | – | – | IANA, e.g. Europe/Berlin. Wins over the mode. |
blockAds | boolean | false | – | Refuse requests to ad and tracker sites inside the machine (its local proxy), with or without a session proxy. Blocked requests are counted in blockedRequests and reported in lifecycle events. |
waitUntil | string | load | load, domcontentloaded, networkidle, commit | |
timeoutMs | integer | – | – | |
delayMs | integer | – | – | Extra wait after the page loads. From 0 to 10000. |
viewport | object | – | – | |
viewport.width | integer | – | – | |
viewport.height | integer | – | – | |
format | string | markdown | markdown, html, text | |
links | boolean | false | – | Also return the page's absolute http(s) links (at most 2000). |
More examples:
{"url":"https://example.com","format":"html","links":true}curl -X POST "https://api.boxline.dev/v1/fetch" \ -H "x-api-key: $BOXLINE_API_KEY" \ -H "content-type: application/json" \ -d '{"url": "https://example.com"}'import { Boxline } from "@boxline/sdk";const bx = new Boxline(); // reads BOXLINE_API_KEY and BOXLINE_API_URLconst page = await bx.fetch("https://example.com");console.log(page.title, page.content);from boxline import Boxlinebx = Boxline() # reads BOXLINE_API_KEY and BOXLINE_API_URLpage = bx.fetch("https://example.com")print(page["title"], page["content"])boxline fetch https://example.comResponses
200 OK
The page.
Headers
RateLimit-Limit: Requests allowed in the current window of this route's limit. (Rate limits)RateLimit-Remaining: Requests left in the current window. (Rate limits)RateLimit-Reset: Seconds until the window starts again. (Rate limits)
Fields
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
url | string | – | – | |
finalUrl | string | – | – | |
status | integer | null | – | – | |
title | string | – | – | |
content | string | – | – | |
captcha | string | null | – | recaptcha, hcaptcha, turnstile, cloudflare, datadome, arkose, human | A CAPTCHA waiting for a person on the page. |
ms | integer | – | – | |
links | string[] | – | – | With links: true. |
{ "url": "https://example.com/", "finalUrl": "https://example.com/", "status": 0, "title": "Example Domain", "content": "…", "captcha": "recaptcha", "ms": 0, "links": [ "…" ]}Errors
Errors have a matching HTTP status and the same body. code is stable, so check it in your code; message is for people. See Errors and request ids.
| Status | Codes | When |
|---|---|---|
| 400 Bad Request | invalid_request, invalid_json, invalid_cursor, invalid_idempotency_key, invalid_extension, variables_with_extensions, terms_not_accepted, invalid_reset_token, invalid_confirmation_code, wrong_password, no_password, invalid_next, invalid_oauth_token | The request is not valid. |
| 401 Unauthorized | unauthorized | Missing or invalid API key. |
| 402 Payment Required | feature_not_in_plan, spend_limit, out_of_credit, plan_limit, spend_limit, out_of_credit, organization, project, plan, credit | The plan does not include this, a monthly allowance or spending limit is used up, the organization has no credit left, or the plan allows no more of this. spend_limit and out_of_credit carry error.details.limit: organization, project, plan or credit. |
| 422 Unprocessable Content | not_a_web_page, curl, pdftotext | The address is a PDF: Chrome only displays it, so there is no page text. Download it in a session (files, or curl in its shell) and read it there, e.g. with pdftotext. |
| 429 Too Many Requests | rate_limited, concurrency_limit | Too many requests, or the organization's concurrent sessions (its plan's) or the project's concurrencyCap are in use. |
| 502 Bad Gateway | machine_error, machine_unreachable, page_unreachable | The session machine failed or could not be reached, or, for fetch, screenshot, pdf and extract, the page could not be loaded. |
| 503 Service Unavailable | no_capacity, proxy_unavailable, provider_unavailable, search_unavailable, webhooks_unavailable, mail_unavailable, browser_unreachable | No machine free in time, proxies not set up, the model provider is not configured or has no key, or web search is not set up or its provider is not answering, or webhook delivery is not set up, or an email cannot be sent, or the session's browser did not answer in time. |
| 504 | page_timeout | The page did not finish loading within timeoutMs. |
{ "error": { "code": "not_found", "message": "session not found", "requestId": "req_4fJx0aQm2Lk9Pz7N" }}Learn more
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.