Docs
Dashboard
SDKs / MCP serverBeta

web_crawl_get

Get a crawl's state and its next pages.

Returns the state of a crawl that web_crawl_start started, and its pages, 10 at a time. Use it when web_crawl_start says the crawl is still running, or to read the next pages. It only reads: it does not start or change a crawl.

Inputs

NameTypeDefaultAllowed valuesDescription
crawlIdrequiredstring––The crawl's id, from web_crawl_start.
afterstring––The cursor from the last result (its last line says after "…"), to read the pages that follow. Leave it out for the first pages.

Example call

This is what an MCP client sends. You do not write it yourself: ask the model in plain words and it calls the tool.

tools/call
{  "name": "web_crawl_get",  "arguments": {    "crawlId": "crawl_2Gh7Kx0Lq9Zp4Nm3"  }}

What it returns

The same report as web_crawl_start: the crawl's status and counts, up to 10 pages, and a last line that either says to call web_crawl_get again or says every page has been read.

Result
Crawl crawl_2Gh7Kx0Lq9Zp4Nm3 of https://example.com/docs: running; 12 pages read, 0 failed, 0 skipped by robots.txt.## Installhttps://example.com/docs/install (HTTP 200, depth 1)…The crawl is still running: call web_crawl_get with crawlId "crawl_2Gh7Kx0Lq9Zp4Nm3" in a few seconds.

When a call fails, the result is marked as an error. Its text starts with the API’s error code, such as unauthorized or session_not_running, followed by its message.

Annotations

The server tells every client what this tool does to the world, so a client can decide what to ask you about before it runs.

AnnotationValue
titleGet a crawl's pages
readOnlyHinttrue
destructiveHintfalse
openWorldHintfalse

Good to know

  • Pass the after that the last result gave to read the next pages. The crawl keeps running in the background while you read.

The same thing with the API

EndpointReference
GET /v1/crawls/{id}Get a crawl

These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.