Core conceptsBeta

Browser

Drive Chrome over CDP with Playwright or Puppeteer, watch it in the live view and run low-latency actions.

Sessions have a Chrome browser unless you create them with "browser": false. The session’s connectUrl is a Chrome DevTools Protocol (CDP) WebSocket, so any CDP client can drive it.

Playwright

TypeScript
import { chromium } from "playwright";

const browser = await chromium.connectOverCDP(session.connectUrl);
const context = browser.contexts()[0];
const page = context.pages()[0] ?? (await context.newPage());

await page.goto("https://example.com");
await page.screenshot({ path: "example.png" });

Puppeteer

TypeScript
import puppeteer from "puppeteer-core";

const browser = await puppeteer.connect({
  browserWSEndpoint: session.connectUrl,
});
const [page] = await browser.pages();
await page.goto("https://example.com");

Raw CDP

You can also speak the protocol directly over a WebSocket, which is useful for tools that do not use Playwright or Puppeteer:

TypeScript
import WebSocket from "ws";

const ws = new WebSocket(session.connectUrl);
ws.on("open", () => {
  ws.send(JSON.stringify({ id: 1, method: "Browser.getVersion" }));
});
ws.on("message", (data) => console.log(JSON.parse(data.toString())));

Disconnecting matters for browser-only sessions: without keepAlive, they end about 5 seconds after the last CDP client leaves. See session lifetime.

Live view

The session’s liveUrl is a page that shows the browser in real time. Open it to watch your agent, or click and type in it to take over when the agent is stuck, for example to solve a CAPTCHA. It is a signed URL: anyone who has it can control the browser, so share it only with people who should. Get a fresh one with GET /v1/sessions/:id/live.

Only Boxline’s own apps, such as the console, can frame the live view. Embedding it in an <iframe> on your own site needs embed origins set for your project, which are not available yet; open it in a new tab or window instead.

Downloads

Whatever triggers a download (Playwright, Puppeteer, the Actions API or a click in the live view), the file lands in /workspace/downloads. Wait for it with files/wait?pattern=downloads/*.csv, process it in the shell, or download it to your own machine. See Files. To reuse the browser’s login from the shell, export its cookies.

Actions API

POST/v1/sessions/:id/actions

The Actions API runs browser steps next to the browser, inside the session. Send one action or a whole list in a single request instead of one round trip per step, which keeps latency low for agents that plan several steps at a time.

Shell
curl -X POST https://api.staging.boxline.dev/v1/sessions/$SESSION_ID/actions \
  -H "x-api-key: $BOXLINE_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "actions": [
      { "action": "goto", "url": "https://example.com/login" },
      { "action": "fill", "selector": "#email", "value": "agent@example.com" },
      { "action": "click", "selector": "button[type=submit]" },
      { "action": "content", "format": "markdown" }
    ]
  }'

Actions run in order. The response has one result per action:

Response (shape)
{
  "results": [
    { "ok": true, "action": "goto", "value": { "url": "…", "title": "…", "status": 200 }, "ms": … },
    { "ok": true, "action": "fill", "ms": … },
    { "ok": true, "action": "click", "ms": … },
    { "ok": true, "action": "content", "value": { "content": "…", "url": "…", "title": "…" }, "ms": … }
  ]
}

A failed action has ok: false, an error message and, when there is one, a stable code (for example captcha_timeout). Available actions:

ActionFieldsWhat it does
gotourl, waitUntil?Navigate the current tab.
clickselector, or x and yClick an element (Playwright selector) or a point on the page.
fillselector, valueSet the value of an input.
typetext, selector?, delayMs?Type text, optionally into an element first.
presskeyPress a key or shortcut, e.g. Enter or Control+A.
scrolldeltaY, x?, y?Scroll the page.
waitselector? or ms?Wait for an element or a fixed time.
screenshotfullPage?, format?, quality?Returns { data (base64), mimeType }.
contentformat: "markdown" | "html" | "text"Returns { content, url, title }.
evaluateexpressionRuns JavaScript in the page; returns { value }.
uploadselector, pathSets a file input to a file inside the sandbox, e.g. from /workspace.
tabs, newTab, switchTab, closeTaburl?, index?List, open, switch and close tabs.
back, forward, reloadnoneHistory navigation.
stepinstruction, variables?, provider?, model?One plain-English instruction, e.g. "click Sign in": a fast model picks one action and the platform runs it. Returns what it did and the matching Playwright line (code). Use %name% for variables; their values never reach the model. While a CAPTCHA waits for a person it waits too, up to 4 minutes, then fails with code captcha_timeout (see CAPTCHAs).

The SDK wraps the common ones: session.goto(url), click(selector), fill(selector, value), content(format) and screenshot(opts) and step(instruction), plus session.actions(list) for everything else. You can mix styles: drive the session with Playwright for complex flows and use actions for quick, latency-sensitive steps.

These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.