AI agent

The AI agent does a task you describe in plain words. It works in a Boxline session, with a browser, a shell or both, until it has an answer. You can watch it in the live view, pause it to act yourself, and resume it.

Goal or steps

There are three ways to get browser work done with AI. Pick by how much control you want:

Start a run

POST /v1/agent/runs starts the run and answers at once with its id. The run then works in the background.

curl -X POST https://api.boxline.dev/v1/agent/runs \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -H "Idempotency-Key: $(uuidgen)" \  -d '{"task": "Open example.com and summarise what the page is for in two sentences."}'

const { id } = await bx.agent.run({  task: "Open example.com and summarise what the page is for in two sentences.",});const run = await bx.agent.wait(id); // polls until it finishesconsole.log(run.status, run.result);// What it did, with the thought behind each step.for (const s of run.steps) console.log(s.type, s.name ?? "", s.thought ?? "");

started = bx.agent.run(task="Open example.com and summarise what the page is for in two sentences.")run = bx.agent.wait(started["id"])  # polls until it finishesprint(run["status"], run["result"])for s in run["steps"]:  # what it did, with the thought behind each step    print(s["type"], s.get("name", ""), s.get("thought", ""))

# Shows each step and thought as it happens; the answer comes last, on stdout.boxline run "Open example.com and summarise what the page is for in two sentences."

Open in the playground ↗

The response (201) is { id, status, sessionId, sessionExpiresAt, provider, model, keySource, mode } with status "running". sessionExpiresAt is when the run’s session ends by its time limit.

The run’s own session

A run works in a session. Unless you give it one with sessionId, it starts a session of its own and stops it when the run ends. session describes that session. It takes the same fields as Create a session, checked by the same code, with the same answers. Leave it out for a browser only.

A session is a browser only, a shell only or both: see The three kinds of session. The agent’s tools follow the session, so the kind you choose is what the agent can do.

• Settings go inside session. A session setting at the top level of the run, next to task, is 400 invalid_request. task, sessionId, variables, credentials, allowWithExtensions, keepSession and the limits stay fields of the run.
• One or the other. session together with sessionId is 400 invalid_request: use one session, or describe a new one. With sessionId, that session keeps its own settings and its own time. Change its length with Update a session.
• How long. A run’s session lasts 1,800 seconds (30 minutes) unless session.timeout says otherwise, or your plan’s maximum when that is shorter. That is so also when you send session without a timeout. Above the plan’s maximum is 402 plan_limit. A run stops when its session’s time ends: see When a run stops.
• Two names, two jobs. The run’s credentials are what the AI may type. session.credentials are exported into the session’s shell for commands, and need a shell.
• With keepSession: true the session keeps running after the run, until its expiresAt (it gets keepAlive).

curl -X POST https://api.boxline.dev/v1/agent/runs \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -H "Idempotency-Key: $(uuidgen)" \  -d '{"task": "Open example.com and summarise what the page is for.", "session": {"blockAds": true, "cookieBanners": "reject"}}'

const run = await bx.agent.run({  task: "Open example.com and summarise what the page is for.",  session: { blockAds: true, cookieBanners: "reject" },});

run = bx.agent.run(    "Open example.com and summarise what the page is for.",    session={"blockAds": True, "cookieBanners": "reject"},  # the API's names, as on Create a session)

Shell only

Give the run a session with no browser, and it works with a shell only. Use it for work that needs no web page: convert files, run a script, go through data in /workspace. The agent’s tools follow the session, so a shell-only run has no browser_* tools. It has bash, and web_search when your plan includes it. mode: "computer" needs a browser, so with browser: false it is 400 invalid_request.

curl -X POST https://api.boxline.dev/v1/agent/runs \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -H "Idempotency-Key: $(uuidgen)" \  -d '{"task": "Convert every video in /workspace to MP4.", "session": {"browser": false, "shell": true}}'

const { id } = await bx.agent.run({  task: "Convert every video in /workspace to MP4.",  session: { browser: false, shell: true },});const run = await bx.agent.wait(id);console.log(run.status, run.result);

started = bx.agent.run(    "Convert every video in /workspace to MP4.",    session={"browser": False, "shell": True},)run = bx.agent.wait(started["id"])print(run["status"], run["result"])

boxline run "Convert every video in /workspace to MP4." --no-browser --shell

A person can still step in: pause the run and work in the session’s terminal. The agent’s shell commands are in the session’s events too, each as an exec event with by: "agent" and the run’s runId. See Session events.

Choose a model

GET /v1/agent/models lists every provider and model you can pick, with prices per million tokens and whether the provider is available. The same browser and shell tools work with every model.

Grok, Gemini and OpenRouter are new: try them on your own tasks first. They do not support mode: "computer". With no provider or model, the run uses your project’s default model.

A provider with no key anywhere answers 503 provider_unavailable. The Free plan runs models on your own keys only: without a key for the provider, 402 feature_not_in_plan. Each run reports usage.costUsd: the model cost of that run, separate from session time, and keySource: whose key it used ("platform", "project", or "run" for a modelKey sent with the run). Model use on Boxline’s keys counts toward your plan’s monthly spend cap; past it, new runs get 402 spend_limit. A run on your own key is not charged by us and is outside that cap. Your organization’s spending limit covers every run, on your key too, because the run’s session time is still usage. If the provider rejects your key, the run fails with model_key_rejected and never switches keys. See Your own model keys.

Follow a run and get the result

GET /v1/agent/runs/:id returns the run: status, steps, result, resultText, output, error, errorCode, usage, handover, and taskId and taskRunId when a task started it. It also shows its limits (maxSteps, maxCostUsd, maxConsecutiveErrors), sessionExpiresAt (when its session stops, for showing the time left), resumable (see Pause and resume a run), variableNames: the names of its variables, never their values, and promptCache when it uses Claude prompt caching (then usage also has cacheReadTokens and cacheWriteTokens).

• status is "running", "paused" (waiting for a person, see below), "completed", "failed" or "canceled".
• result holds the final answer: text, or JSON when the run has an output schema. resultText is the final text (with a schema, a one-sentence summary).
• error gives the reason for a failure. errorCode is a stable code for it: one of the limits in When a run stops, output_invalid (see Structured output), internal (an error on the platform’s side; error then says only that), or the code of the error that stopped the run, such as spend_limit, captcha_timeout, handover_timeout or machine_unreachable. It is null otherwise.
• steps lists what happened, in order: tool (a tool call, with name, input, output, ms, thought and, where they apply, the n, screen, shell and usage of What a step shows), handover and handback (when the run paused for a person, and when it went on), captcha (see CAPTCHA steps), code (see Code steps), and a final text step with the agent’s answer.
• Steps are kept as long as the session’s logs: 7, 30 or 90 days after the session stops, by plan. After that the run keeps its task, result and token counts. See How long they are kept.

Thoughts

With each tool call, the agent writes one short sentence saying what it is doing and what it will do next. That sentence is the step’s thought, such as “The search results are open; I’ll check the second result, which looks like the pricing page.” A handover step that the agent asked for (with ask_user_for_help) has one too. With several calls in one reply, each gets the text written before it.

• A thought is hidden like everything else the run keeps: a variable’s value becomes its %name%. It is at most 2,000 characters.
• When the provider gives the model’s reasoning in words, it is part of the thought: OpenAI’s reasoning summaries, and Claude’s thinking when a reply has it. So a computer-use step, which often has no text of its own, still shows what the model was thinking. Markdown bold is removed. Encrypted or redacted reasoning is never kept.
• The model’s final answer stays a text step. Text that comes with tool calls is no longer a separate text step: it is their thought.

What a step shows

A run reads as a story: what the agent saw, what it thought, what it did, what came out and what it cost. Each step carries the parts that apply to it:

curl https://api.boxline.dev/v1/agent/runs/$RUN_ID \  -H "x-api-key: $BOXLINE_API_KEY"

const run = await bx.agent.get(runId);for (const s of run.steps) {  console.log(s.n, s.name ?? s.type, "|", s.thought ?? "");  console.log("  ", s.output ?? s.text ?? "");  if (s.screen?.image) console.log("   screen:", s.screen.title, s.screen.image);  if (s.shell) console.log("   shell: exit", s.shell.exitCode, s.shell.files?.map((f) => f.path));  if (s.usage) console.log("   cost:", s.usage.inputTokens, "in,", s.usage.outputTokens, "out,", s.usage.costUsd);}

run = bx.agent.get(run_id)for s in run["steps"]:    print(s.get("n"), s.get("name", s["type"]), "|", s.get("thought", ""))    print("  ", s.get("output") or s.get("text", ""))    if s.get("screen", {}).get("image"):        print("   screen:", s["screen"]["title"], s["screen"]["image"])    if "shell" in s:        print("   shell: exit", s["shell"]["exitCode"], [f["path"] for f in s["shell"].get("files", [])])    if "usage" in s:        print("   cost:", s["usage"]["inputTokens"], "in,", s["usage"]["outputTokens"], "out,", s["usage"]["costUsd"])

What a step’s output says

A tool step’s output is what the model was told, and it reads well to a person too. A click, double-click, right-click, key press or choice names the element it acted on and the page the browser is on afterwards:

Clicked button "Log in". Page: Dashboard — https://example.com/dashboardTyped into textbox "Email".Selected Germany in list "Country". Page: Checkout — https://example.com/checkoutPressed Enter. Page: Search results — https://example.com/search?q=boxes

• An element is named by its role and visible name: link "Pricing", button "Log in", textbox "Email". Names are cut at 60 characters. When the element has no name, only its role is given; when it cannot be found, the output says it.
• A click by position names what is there, never the value of a field: Clicked “Jobs” link at (814, 124). A computer-use step that took the tab to another page ends with → example.com/jobs, and an address typed into the address bar (Ctrl or Cmd+L, the address, Enter, in one call) reads Opened example.com/jobs.
• Typing never repeats what was typed, so a password or a %variable% value does not appear in the output.
• After a click or a key press, the output waits a moment (up to 3 seconds) for a page the action started to load, then names its title and address.

GET /v1/agent/runs/:id/events/stream streams the run as server-sent events, as it happens: one event per step, a thought event ({ type: "thought", text, at }) as soon as the model’s reply arrives, before its tool runs, a status event when it pauses or resumes, exec and output events while a shell command runs, and a final done event with the result. A late viewer first gets the running command and its output so far. The stream stays open while the run is paused.

curl -N https://api.boxline.dev/v1/agent/runs/$RUN_ID/events/stream \  -H "x-api-key: $BOXLINE_API_KEY"

for await (const event of bx.agent.stream(runId)) {  if (event.type === "thought") console.log("…", event.text); // what it is about to do  else if (event.type === "tool") console.log("→", event.name, JSON.stringify(event.input));  else if (event.type === "done") console.log(event.status, event.result); // the last event}

for event in bx.agent.stream(run_id):    if event["type"] == "thought":        print("…", event["text"])  # what it is about to do    elif event["type"] == "tool":        print("→", event["name"], event.get("input"))    elif event["type"] == "done":        print(event["status"], event.get("result"))  # the last event

POST /v1/agent/runs/:id/cancel stops a run for good. It interrupts a model call in progress and stops the run’s own session. GET /v1/agent/runs lists your runs, newest first.

Structured output: answers as JSON

When your code reads the answer, ask for JSON. Pass output, a JSON Schema, and the run’s result is JSON that matches it. Tasks take the same option, so every run of a task returns the same shape.

curl -X POST https://api.boxline.dev/v1/agent/runs \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -H "Idempotency-Key: $(uuidgen)" \  -d '{    "task": "Open https://books.toscrape.com and list the books on its first page with their prices.",    "output": {      "type": "object",      "properties": {        "books": {          "type": "array",          "items": {            "type": "object",            "properties": {"title": {"type": "string"}, "price": {"type": "number"}},            "required": ["title", "price"]          }        }      },      "required": ["books"]    }  }'# Once "status" is "completed", "result" is the JSON answer:curl https://api.boxline.dev/v1/agent/runs/$RUN_ID -H "x-api-key: $BOXLINE_API_KEY"

const { id } = await bx.agent.run({  task: "Open https://books.toscrape.com and list the books on its first page with their prices.",  output: {    type: "object",    properties: {      books: {        type: "array",        items: {          type: "object",          properties: { title: { type: "string" }, price: { type: "number" } },          required: ["title", "price"],        },      },    },    required: ["books"],  },});const run = await bx.agent.wait<{ books: { title: string; price: number }[] }>(id);if (run.status === "completed" && run.result) {  const { books } = run.result;  console.log(run.resultText, books.length, books[0]);} else {  console.log(run.errorCode, run.error); // "output_invalid" when the answer did not match}

started = bx.agent.run(    task="Open https://books.toscrape.com and list the books on its first page with their prices.",    output={        "type": "object",        "properties": {            "books": {                "type": "array",                "items": {                    "type": "object",                    "properties": {"title": {"type": "string"}, "price": {"type": "number"}},                    "required": ["title", "price"],                },            },        },        "required": ["books"],    },)run = bx.agent.wait(started["id"])if run["status"] == "completed":    books = run["result"]["books"]    print(run["resultText"], len(books), books[0])else:    print(run["errorCode"], run["error"])  # "output_invalid" when the answer did not match

{  "status": "completed",  "result": { "books": [{ "title": "A Light in the Attic", "price": 51.77 }, "…"] },  "resultText": "Listed the 20 books on the first page with their prices.",  "output": { "type": "object", "…": "…" },  "error": null,  "errorCode": null}

How the answer is made

• The model gets a finish tool whose input is your schema, and is told to call it once with the answer. A final text answer that is JSON matching the schema (bare, or in a ```json block) is taken too.
• A schema that is not an object at the top, such as a list, is wrapped as { "result": … } for the model and unwrapped again. Your result is still the list.
• resultText is the model’s one-sentence summary, or “Returned {…}” when it wrote none (up to 500 characters). output is the schema the run used.
• Each finish call is a tool step named finish, with the answer as input, and isError when it did not match.
• The stream’s final done event and the agent_run.finished webhook carry the JSON too, with structured: true.
• Allow null for values a page may not have ("type": ["number", "null"], or "nullable": true). The model is told to use null there rather than invent a value.

When the answer does not match

1. The model gets one repair try. It is told what is wrong, such as $.books[0].price: must be number, got string, and answers again.
2. If the new answer does not match either, the run fails: status: "failed", errorCode: "output_invalid", and error lists the problems (up to 20).

An answer that needs more checking than a run allows (see Limits) fails at once with output_invalid (“the output schema is too complex to check an answer against”), without a repair try.

Supported keywords

Limits

• At most 32 KB (as JSON), 2,000 parts and 32 levels deep.
• $ref only to places inside the schema ("#/$defs/…", "#/definitions/…"), and each must point to a schema there: not into enum, const, default or a keyword the platform does not know.
• type lists without repeats, and required and enum of at most 1,000 entries.
• No U+0000 character and no lone UTF-16 surrogate.
• A $ref loop that never goes into a property or an item is refused, such as { "anyOf": [{ "$ref": "#" }] }. A tree through properties or items is fine.
• A schema too complex to check answers against, with choices and references that multiply the work, is refused too.
• Checking one answer takes at most 50,000 steps, 10,000 $ref uses, a $ref depth of 64 and 50 milliseconds.

The schema is checked when you start the run, or save the task. A schema that breaks a rule gets 400 invalid_request, with the place named: “output is not a valid JSON Schema: $.properties.price.type: …”.

Secrets in answers

The answer is hidden like everything else the run keeps. After it is checked against the schema, every variable’s and credential’s value in it, in keys too, becomes its placeholder. So do not ask for a secret in the output: you get "%password%" back. Values shorter than 3 characters are not hidden. A U+0000 character or a lone surrogate in an answer becomes U+FFFD.

When a run stops

A run stops at the first of its limits, its session’s time, or a cancel. The limits are checked before each model call, and waiting for a person uses time, not steps. The first five below can be resumed. The whole picture, with the sessions, is on How runs and sessions end.

A run without a step limit (maxSteps: null) goes until it is done, reaches another limit, or runs out of session time: give it a maxCostUsd or a sensible session.timeout.

curl -X POST https://api.boxline.dev/v1/agent/runs \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -H "Idempotency-Key: $(uuidgen)" \  -d '{"task": "Convert every video in the workspace to MP4.", "session": {"shell": true, "timeout": 3600}, "maxSteps": null, "maxCostUsd": 2}'

const { id } = await bx.agent.run({  task: "Convert every video in the workspace to MP4.",  session: { shell: true, timeout: 3600 },  maxSteps: null, // no step limit: the cost limit and the session's time bound it  maxCostUsd: 2,});const run = await bx.agent.wait(id);console.log(run.status, run.errorCode, run.resumable);

started = bx.agent.run(    task="Convert every video in the workspace to MP4.",    session={"shell": True, "timeout": 3600},    max_steps=None,  # no step limit: the cost limit and the session's time bound it    max_cost_usd=2,)run = bx.agent.wait(started["id"])print(run["status"], run["errorCode"], run["resumable"])

boxline run "Convert every video in the workspace to MP4." --shell --steps none --max-cost 2 --timeout 3600

Pause and resume a run

Two calls steer a run. Pause holds it so a person can act. Resume carries it on. Resume is one call for two cases, and in both it is the same run: one run id, and its steps and usage add up. The full requests and answers are in the API reference: Pause a run and Resume a run.

Pause a run

POST /v1/agent/runs/:id/pause takes an optional reason. The agent finishes its current action, then waits, and the run shows status: "paused". Now a person can act in the live view or the session’s terminal: solve a CAPTCHA, type a 2FA code, make a choice. Pause works on every kind of run: a browser only, a shell only or both.

• Only a running run can be paused: otherwise 409 invalid_state. 409 run_not_live means the server running it stopped: see below.
• handover on the run says who asked and why: see When a person is needed.
• A paused run waits up to 30 minutes. Without a resume it fails with errorCode: "handover_timeout".
• The agent_run.waiting webhook goes out when a run waits for a person, so you need not poll.

Resume a run

POST /v1/agent/runs/:id/resume answers 200 with the run, working again (status: "running"). What you can send depends on why the run is not working:

curl -X POST https://api.boxline.dev/v1/agent/runs/$RUN_ID/pause \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -d '{"reason": "I will sign in myself"}'# … a person signs in through the live view …curl -X POST https://api.boxline.dev/v1/agent/runs/$RUN_ID/resume \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -d '{"note": "Signed in. Carry on from the dashboard."}'

await bx.agent.pause(runId, "I will sign in myself");// … a person signs in through the live view …await bx.agent.resume(runId, { note: "Signed in. Carry on from the dashboard." });

bx.agent.pause(run_id, reason="I will sign in myself")# … a person signs in through the live view …bx.agent.resume(run_id, note="Signed in. Carry on from the dashboard.")

# Resumes a paused run and watches it, like boxline run.boxline resume $RUN_ID --note "Signed in. Carry on from the dashboard."

Resume a run that stopped early

A run that stopped at one of its limits (max_steps, max_cost, too_many_errors, no_progress) or because its server restarted (server_restarted) keeps its progress and its session. Resume it, and the same run goes on in the same session, with the same model, mode, output schema, credentials and browser profile. The agent is told what the run did so far, so it does not redo finished work.

curl -X POST https://api.boxline.dev/v1/agent/runs/$RUN_ID/resume \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -H "Idempotency-Key: $(uuidgen)" \  -d '{"maxSteps": 50, "note": "The download is done; do the upload next."}'

const run = await bx.agent.get(runId);if (run.resumable) {  await bx.agent.resume(runId, {    maxSteps: 50,    note: "The download is done; do the upload next.",  });  const done = await bx.agent.wait(runId); // the same run, going on  console.log(done.status, done.result);}

run = bx.agent.get(run_id)if run.get("resumable"):    bx.agent.resume(run_id, max_steps=50, note="The download is done; do the upload next.")    done = bx.agent.wait(run_id)  # the same run, going on    print(done["status"], done["result"])

# Watches the run like boxline run.boxline resume $RUN_ID --steps 50 --note "The download is done; do the upload next."

• The window: resumable: { until } is set while the run can be resumed: 10 minutes, or less when its session’s own time ends sooner. A run’s own session is kept that long for it, and billed like any running session. When the window passes unused, the session stops.
• The body: maxSteps (1 to 1,000 or null: the steps of this stretch; default the run’s own), maxCostUsd (0.01 to 100 or null: the model cost this stretch may add; default the run’s own), note (one text for the AI, up to 2,000 characters: what a person did after a pause, or what to do next after a limit) and variables.
• Variables are sent again. Their values are never stored, so a run that had variables needs every name again (variableNames lists them), as text or { "value": … }. Each keeps the sites and shell rule it had: sending origins or shell here is 400. A missing name is 400 missing_variables, and a name the run did not have is 400 too.
• One run. The answer is the same run. steps and usage hold the whole run, and error, errorCode, finishedAt and resumable are cleared while it works. It can stop at a limit again and be resumed again. The webhook agent_run.resumed goes out, and agent_run.finished again when it ends. A run that a task started can be resumed too: its task run goes back to running.
• Refusals: 409 not_resumable when the run is neither paused nor stopped early (it is running, completed, canceled, failed for another reason, or its window passed), 409 session_not_running when its session was deleted, and 409 run_not_live for a run whose server stopped but that still shows as running or paused (within about 90 seconds it shows as server_restarted; resume it then). 409 session_ended also comes when its session ended with the server: start a new run.
• The usual checks apply again: your plan’s agent runs (402), the spend cap (402 spend_limit), the 30 runs a minute, and a suspended project (403). No new session is taken. A stopped session is resumed first (429 concurrency_limit). An Idempotency-Key makes a retry safe: variable values are not compared.

Send a message to a working run

Tell the agent something while it works, without pausing it: “Also open page C”, “use the second address”. It reads the message at its next step, before its next model call; a model call or a tool already under way is not interrupted.

curl -X POST https://api.boxline.dev/v1/agent/runs/$RUN_ID/messages \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -d '{"text": "Also open page C and include its heading."}'# 202 {"id": "…", "at": "…", "delivered": false}

const sent = await bx.agent.sendMessage(runId, "Also open page C and include its heading.");console.log(sent.id, sent.delivered); // false now; it becomes a message step when the model reads it

sent = bx.agent.send_message(run_id, "Also open page C and include its heading.")print(sent["id"], sent["delivered"])  # False now; it becomes a message step when the model reads it

boxline message $RUN_ID "Also open page C and include its heading."# While boxline run watches a run, type a line and press Enter to send it.

• It answers the agent’s questions. A run paused because the agent asked for help (handover.by: "agent") takes the message as the answer and goes on at once.
• Not for other pauses. Runs you paused, runs paused for a CAPTCHA, and runs paused for OpenAI’s safety checks stay paused; their messages are delivered after you resume them, and only a resume acknowledges safety checks. A safety-check pause also shows handover.by: "agent", with the checks as its reason: read the reason before you answer.
• Recorded, and hidden like the rest. A delivered message becomes a message step, and appears in the run’s event stream. Variable and credential values in it become their placeholders. Type passwords and codes into the page (the live view), never into a message.
• Limits: 1 to 2,000 characters; 50 messages per run (409 too_many_messages); 10 a minute per run and 60 a minute per project. Once the run has finished it is 409 invalid_state: for a run that stopped at a limit, resume it with a note instead. When the server running it stopped it is 409 run_not_live.
• Who may send them: anyone with your project’s API key or console login. The agent treats a message as your instruction, and never takes the same words from page text, files or command output as one.

Crashed and hung tabs

A heavy page can crash its tab, or stop it from answering. When a browser tool finds its tab crashed, or hung (two actions in a row timed out on it), the platform closes the dead tab and opens a fresh one on the same address. Nothing else runs, and the agent is told: “the tab crashed; a new tab was opened at <url>. Check the page and go on.”

• The session’s log gets a lifecycle event tab_crashed or tab_hung, with the new tab’s url.
• In a run these count toward maxConsecutiveErrors only from the third replaced tab on, when it keeps happening.
• Outside a run, an action on such a tab answers ok: false with code: "tab_crashed" or "tab_hung": it did not run, and you can send it again on the new tab.

Variables: secrets the model never sees

To sign in, the agent needs a password, but the model should never see it. Pass values as variables and refer to them in the task as %name%. The model is told the names only. The platform fills in the value where the model types it.

curl -X POST https://api.boxline.dev/v1/agent/runs \  -H "x-api-key: $BOXLINE_API_KEY" \  -H "content-type: application/json" \  -d '{    "task": "Sign in to example.com with %email% and %password%, then download the latest invoice.",    "variables": {      "email": "ada@example.com",      "password": { "value": "'"$EXAMPLE_PASSWORD"'", "origins": ["https://example.com"] },      "api_token": { "value": "'"$EXAMPLE_TOKEN"'", "shell": true }    }  }'

await bx.agent.run({  task: "Sign in to example.com with %email% and %password%, then download the latest invoice.",  variables: {    email: "ada@example.com",    password: { value: process.env.EXAMPLE_PASSWORD!, origins: ["https://example.com"] },    api_token: { value: process.env.EXAMPLE_TOKEN!, shell: true },  },});

import osbx.agent.run(    task="Sign in to example.com with %email% and %password%, then download the latest invoice.",    variables={        "email": "ada@example.com",        "password": {"value": os.environ["EXAMPLE_PASSWORD"], "origins": ["https://example.com"]},        "api_token": {"value": os.environ["EXAMPLE_TOKEN"], "shell": True},    },)

For a value you use again and again, store it once as a credential and pass credentials: ["NAME"] instead: the same rules apply, and your code never handles the value. For a site’s sign-in, save a password credential (user name, password and where its 2FA codes come from) and write %NAME.username%, %NAME.password% and %NAME.otp%. Link it to a browser profile and every run that starts from the profile gets it without being asked.

Each variable is a value, or an object with more rules:

Where a value can go

• Only into text the agent types in a field, or an option it picks in a list. With shell: true, also into a shell command.
• Never into a URL, a selector, a key press or anything else. A tool call that tries is refused before anything runs, and the model only hears the placeholder’s name.
• With origins, the value is typed only when the field’s own frame is on one of those sites, checked right before it is written. Otherwise the call fails with a message like “%password% can only be used on example.com”. Such a value always needs a field: it is never typed wherever the cursor happens to be.
• A %word% that names no variable is left as it is.

Hidden everywhere

Everything that goes back to the model or is kept has each value replaced by %name%: page text, tool results, shell output, the task, the steps, the result, errors and live events. Values are also found when they are JSON-escaped, percent-encoded or spread over several pieces of output. Screenshots for the model cover every field that holds a value.

While a run with variables works in a session, that session is more careful:

• its event log keeps no query strings in URLs, and no console or page-error text;
• its recording pauses;
• its checkpoints and stops keep no typed form value that holds a variable, and no values in page URLs or titles;
• its checkpoints and stops store the browser without its history, the tabs to restore, autofill and saved passwords. Tabs reopen at their addresses without query strings. See Stop and resume.

Limits: up to 50 variables; names are letters, digits and _, not starting with a digit, 64 characters at most; each value up to 8,000 characters, and 64 KB together. Anything else gets 400 invalid_request. Their values are never stored, so to resume a run that stopped early, send its variables again.

When a person is needed

A run pauses (status: "paused") while it waits for a person. handover says who asked and why; it is null while the run is not paused. The person works in the session’s live view (a browser) or terminal (a shell).

A resume can carry a note for the agent. When the run goes on, the agent is told that a person had control, reads the note, and checks the page again before it does anything else. In the SDKs: bx.agent.pause(id, reason), resume(id, { note }) and cancel(id) (Python: pause, resume, cancel). The examples are under Pause and resume a run.

CAPTCHA steps

When a run pauses for a CAPTCHA, its handover step (by: "captcha") is followed by captcha steps that say what happened. They also arrive live on the event stream.

{ "type": "captcha", "state": "solving", "kind": "recaptcha", "host": "example.com", "at": "…" }{ "type": "captcha", "state": "waiting", "kind": "recaptcha", "host": "example.com", "reason": "…", "at": "…" }{ "type": "captcha", "state": "solved", "by": "person", "ms": 41200, "at": "…" }

• "solving": automatic solving is on it (captcha: "solve"). Nobody needs to act.
• "waiting": a person’s turn. reason says why, when solving was refused or failed.
• "solved": the CAPTCHA is gone and the run goes on. by is "auto" or "person", and ms is how long the run waited. A resume by hand counts as solved only when the CAPTCHA is really gone.

Code steps

When a run waits for a password’s 2FA code or sign-in link (its credential’s codeSource is "push" or "url"), it records code steps. They also arrive live on the event stream. They never hold the code or the link.

{ "type": "code", "state": "waiting", "credential": "SHOP", "kind": "code", "text": "Waiting up to 300 s for the 2FA code the site sent for SHOP", "at": "…" }{ "type": "code", "state": "received", "credential": "SHOP", "kind": "code", "at": "…" }

• "waiting": the run waits now, so send the code or link. kind says which: "code" or "link". The webhook credential.code_needed goes out at the same time.
• "received": it arrived, and the run goes on.
• "timeout": nothing arrived within the credential’s codeTimeoutSeconds. The tool call fails with credential_code_timeout.

See 2FA codes and sign-in links for how to send them.

What the agent can do

The agent works through tools. Its tool calls are in steps, with their name.

The tools follow the session. In a session without a browser, the agent has no browser_* tools: see Shell only. In computer mode, the page tools are replaced by the provider’s computer-use tool.

Writing good tasks

• Say what done looks like: the answer, file or state you expect.
• Name the site or starting URL when you know it.
• Keep one goal per run; chain runs for work in several parts.
• Put passwords and tokens in variables or credentials, never in the task text.
• When code reads the answer, ask for JSON with output.
• Run the same work again and again? Save it as a task, with variables for what changes and a schedule if it should run on its own.

Limits

• 30 new runs per minute per project, resumes of runs that stopped early and task runs included; messages 10 a minute per run. See Rate limits.
• Your plan must include agent runs (402 feature_not_in_plan).
• A run whose server stops (a restart or a crash) ends as server_restarted, at once or within about 90 seconds, while its session goes on: resume it.