Run a computer-use action
Runs ONE action exactly as a computer-use model's tool gives it, on the session's browser page (never the machine's desktop), then returns the screen: a screenshot, the pointer, URL and title. Two shapes: one action of Claude's computer toolset (computer_toolset_20260801) as {action: <member name>, ...input}, e.g. {action:"left_click", coordinate:[x, y]}, and one item of an OpenAI computer_call's actions (GA computer tool: {type:"click", x, y, button}). Coordinates are pixels of the screenshot: with maxWidth the screenshot is scaled down and coordinates are divided by scale. A failure in the page is 200 with ok:false (and still a screenshot); input that cannot be mapped, or a point outside the screenshot, is 400 (invalid_request, out_of_viewport). 300 calls per minute per session; screenshots also count against the session's 300 per minute. 409 browser_disabled on a session created with browser: false.
POST
/v1/sessions/{id}/browser/computerRequest
| Method and path | POST/v1/sessions/{id}/browser/computer |
|---|---|
| Base URL | https://api.boxline.dev |
| Authentication | Your API key in the x-api-key header (or Authorization: Bearer). The examples read it from BOXLINE_API_KEY. |
| Rate limit | Yes. See Rate limits for the numbers and headers. |
Path parameters
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
idrequired | string | – | – |
Body
JSON with content-type: application/json. Required.
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
actionrequired | string | – | screenshot, cursor_position, mouse_move, left_click, right_click, middle_click, double_click, triple_click, left_mouse_down, left_mouse_up, left_click_drag, scroll, key, hold_key, type, wait, zoom | |
coordinate | number[] | – | – | 2 to 2 items. |
start_coordinate | number[] | – | – | left_click_drag's start. 2 to 2 items. |
text | string | – | – | type: the text; key/hold_key: keys (xdotool style, e.g. ctrl+s); clicks and scroll: keys held. |
key | string | – | – | Keys held during a click. |
scroll_direction | string | – | up, down, left, right | |
scroll_amount | integer | 3 | – | Wheel clicks of 100 px. From 0 to 500. |
duration | number | – | – | Seconds: wait (at most 30), hold_key (at most 10). From 0 to 30. |
repeat | integer | – | – | key: how many times to press the keys (default 1). From 1 to 100. |
region | number[] | – | – | zoom: [x0, y0, x1, y1]. 4 to 4 items. |
maxWidth | integer | – | – | Scale the screenshot down to at most this width; the action's coordinates are read in its pixels. From 100 to 3840. |
screenshot | boolean | true | – | |
format | string | png | png, jpeg | |
quality | integer | – | – | From 1 to 100. |
cursor | boolean | – | – | Draw the pointer on the screenshot. |
More examples:
{ "action": "left_click_drag", "start_coordinate": [ 90, 100 ], "coordinate": [ 200, 100 ], "maxWidth": 640}{ "action": "scroll", "coordinate": [ 640, 360 ], "scroll_direction": "down", "scroll_amount": 3}{"type":"click","x":512,"y":300,"button":"left"}{ "type": "drag", "path": [ { "x": 180, "y": 200 }, { "x": 260, "y": 200 }, { "x": 340, "y": 200 } ]}{"type":"keypress","keys":["CTRL","A"],"screenshot":false}curl -X POST "https://api.boxline.dev/v1/sessions/$SESSION_ID/browser/computer" \ -H "x-api-key: $BOXLINE_API_KEY" \ -H "content-type: application/json" \ -d '{"action": "left_click", "coordinate": [512, 300], "maxWidth": 1024}'import { Boxline } from "@boxline/sdk";const bx = new Boxline(); // reads BOXLINE_API_KEY and BOXLINE_API_URL// The model's action as it is (here Anthropic's shape), and the options.const screen = await bx.sessions.computer("SESSION_ID", { action: "left_click", coordinate: [512, 300] }, { maxWidth: 1024 });console.log(screen.ok ? screen.text : screen.error, screen.cursor, screen.scale);// screen.screenshot (base64) goes back to the modelfrom boxline import Boxlinebx = Boxline() # reads BOXLINE_API_KEY and BOXLINE_API_URL# The model's action as it is (here Anthropic's shape), and the options.screen = bx.sessions.computer("SESSION_ID", {"action": "left_click", "coordinate": [512, 300]}, max_width=1024)print(screen["text"] if screen["ok"] else screen["error"], screen["cursor"], screen["scale"])# screen["screenshot"] (base64) goes back to the modelResponses
200 OK
What happened and the screen after it.
Headers
RateLimit-Limit: Requests allowed in the current window of this route's limit. (Rate limits)RateLimit-Remaining: Requests left in the current window. (Rate limits)RateLimit-Reset: Seconds until the window starts again. (Rate limits)
Fields
| Name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
ok | boolean | – | – | |
action | string | – | – | The provider's action name. |
shape | string | – | anthropic, openai | |
text | string | – | – | One line saying what happened. |
error | string | – | – | |
code | string | – | – | |
screenshot | string | null | – | – | Base64 image (null with screenshot:false). |
mimeType | string | null | – | – | |
width | integer | null | – | – | |
height | integer | null | – | – | |
scale | number | – | – | Screenshot pixels per CSS pixel. |
cursor | object | – | – | CSS pixels of the viewport, from its top-left corner. |
cursor.x | number | – | – | At least 0. |
cursor.y | number | – | – | At least 0. |
url | string | – | – | |
title | string | – | – |
{ "ok": false, "action": "…", "shape": "anthropic", "text": "…", "error": "…", "code": "…", "screenshot": "…", "mimeType": "…", "width": 0, "height": 0, "scale": 0, "cursor": { "x": 0, "y": 0 }, "url": "https://example.com/", "title": "Example Domain"}Errors
Errors have a matching HTTP status and the same body. code is stable, so check it in your code; message is for people. See Errors and request ids.
| Status | Codes | When |
|---|---|---|
| 400 Bad Request | invalid_request, invalid_json, invalid_cursor, invalid_idempotency_key, invalid_extension, variables_with_extensions, terms_not_accepted, invalid_reset_token, invalid_confirmation_code, wrong_password, no_password, invalid_next, invalid_oauth_token | The request is not valid. |
| 401 Unauthorized | unauthorized | Missing or invalid API key. |
| 404 Not Found | not_found, invite_not_found, provider_unavailable | No such thing in this project (or organization), or no such route. |
| 409 Conflict | session_not_running, shell_disabled, browser_disabled, nothing_saved, not_resumable, invalid_state, conflict, email_taken, identity_in_use, provider_already_linked, last_sign_in_method, not_impersonating, machine_too_old, too_many_credential_values, limit_reached, terms_version_mismatch, no_project, last_owner, organization_not_empty, project_has_sessions, already_member, invite_pending, idempotency_in_progress | The current state does not allow it, or a request with this Idempotency-Key is still being answered. |
| 429 Too Many Requests | rate_limited, concurrency_limit | Too many requests, or the organization's concurrent sessions (its plan's) or the project's concurrencyCap are in use. |
| 503 Service Unavailable | no_capacity, proxy_unavailable, provider_unavailable, search_unavailable, webhooks_unavailable, mail_unavailable, browser_unreachable | No machine free in time, proxies not set up, the model provider is not configured or has no key, or web search is not set up or its provider is not answering, or webhook delivery is not set up, or an email cannot be sent, or the session's browser did not answer in time. |
{ "error": { "code": "not_found", "message": "session not found", "requestId": "req_4fJx0aQm2Lk9Pz7N" }}Learn more
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.