Plain-English and exact steps together
Mix exact actions and plain-English sentences in one list of actions, and read what each one did.
One list of actions can hold exact actions and plain-English steps side by side. Write the exact action when you know the selector or the point, and a sentence when you do not, or when the page changes often. They run in order, in one request, next to the browser. They need a session with a browser: a shell-only session answers 409 browser_disabled.
A sentence is a step
In the actions list of POST /v1/sessions/:id/browser/actions, a plain string is a plain-English step: "click Sign in" is the same as { "action": "step", "instruction": "click Sign in" }. A fast model looks at the page and picks one move for it. See Plain-English steps for how that works.
curl -X POST https://api.boxline.dev/v1/sessions/$SESSION_ID/browser/actions \ -H "x-api-key: $BOXLINE_API_KEY" \ -H "content-type: application/json" \ -d @- <<'JSON'{ "actions": [ {"action": "goto", "url": "https://books.toscrape.com"}, "open the Poetry category", {"action": "click", "selector": "article.product_pod h3 a"}, "add this book to the basket if there is a button for it", {"action": "content", "format": "text"} ]}JSONimport { Boxline } from "@boxline/sdk";const bx = new Boxline(); // reads BOXLINE_API_KEY and BOXLINE_API_URLconst results = await bx.sessions.actions("SESSION_ID", [ { action: "goto", url: "https://books.toscrape.com" }, "open the Poetry category", { action: "click", selector: "article.product_pod h3 a" }, "add this book to the basket if there is a button for it", { action: "content", format: "text" },]);for (const r of results) console.log(r.action, r.ok ? r.text : r.error, r.value ?? "");from boxline import Boxlinebx = Boxline() # reads BOXLINE_API_KEY and BOXLINE_API_URLresults = bx.sessions.actions("SESSION_ID", [ {"action": "goto", "url": "https://books.toscrape.com"}, "open the Poetry category", {"action": "click", "selector": "article.product_pod h3 a"}, "add this book to the basket if there is a button for it", {"action": "content", "format": "text"},])for r in results: print(r["action"], r.get("text") if r["ok"] else r.get("error"), r.get("value", ""))Plain-English steps need your plan’s steps feature (402 feature_not_in_plan without it), wherever they are in the list. Exact actions do not.
What each one did
Every result has a text line saying what happened. For an exact action it is the platform’s own line, such as “Opened https://example.com/ (HTTP 200)” or “Double-clicked at (190, 112)”. For a step it is the model’s own description of the move it chose. It never repeats text that was typed or filled in, or a chosen option (“Typed 8 characters”), so it is safe to log.
{ "results": [ { "ok": true, "action": "goto", "text": "Opened https://books.toscrape.com/ (HTTP 200)", "value": { "…": "…" }, "ms": 900 }, { "ok": true, "action": "step", "text": "…", "value": { "method": "click", "code": "…", "…": "…" }, "ms": 1400 }, { "ok": true, "action": "click", "text": "…", "ms": 80 }, { "ok": true, "action": "step", "text": "…", "value": { "…": "…" }, "ms": 1200 }, { "ok": true, "action": "content", "text": "…", "value": { "content": "…", "url": "…", "title": "…" }, "ms": 40 } ]}A step’s value also has the move as a Playwright line (code): collect the lines of a good run and you have a script that runs again without AI. See Scripts.
Secrets in steps
To type a secret, use the object form of a step with variables, and write %name% in the instruction. The value is filled in on the server and never sent to the model. The session remembers it for later steps, so a later sentence can use %email% too.
curl -X POST https://api.boxline.dev/v1/sessions/$SESSION_ID/browser/actions \ -H "x-api-key: $BOXLINE_API_KEY" \ -H "content-type: application/json" \ -d @- <<'JSON'{ "actions": [ {"action": "goto", "url": "https://example.com/login"}, { "action": "step", "instruction": "type %email% into the email field", "variables": {"email": "ada@example.com"} }, "click Sign in", {"action": "wait", "selector": "text=Dashboard"} ]}JSONimport { Boxline } from "@boxline/sdk";const bx = new Boxline(); // reads BOXLINE_API_KEY and BOXLINE_API_URLconst results = await bx.sessions.actions("SESSION_ID", [ { action: "goto", url: "https://example.com/login" }, { action: "step", instruction: "type %email% into the email field", variables: { email: "ada@example.com" }, }, "click Sign in", { action: "wait", selector: "text=Dashboard" },]);for (const r of results) console.log(r.action, r.ok ? r.text : r.error, r.value ?? "");from boxline import Boxlinebx = Boxline() # reads BOXLINE_API_KEY and BOXLINE_API_URLresults = bx.sessions.actions("SESSION_ID", [ {"action": "goto", "url": "https://example.com/login"}, { "action": "step", "instruction": "type %email% into the email field", "variables": {"email": "ada@example.com"}, }, "click Sign in", {"action": "wait", "selector": "text=Dashboard"},])for r in results: print(r["action"], r.get("text") if r["ok"] else r.get("error"), r.get("value", ""))Step variables have no origins limit, so make sure the page is the right one before a step types a password. They are never filled into exact actions, mouse or key actions. See Variables in steps.
When one fails
- The list stops at the first action that fails. Its result has
ok: false, anerrorand, when there is one, a stablecode; the actions after it do not run. - A step waits for a CAPTCHA that needs a person when the session’s
captchaoption is"ask"(the default), and fails withcode: "captcha_timeout"if nobody solves it in time. See CAPTCHAs. - Exact actions are checked before anything runs: a bad shape or a point outside the page is a 400 for the whole list. See Mouse and keyboard.
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.