Starts at one page and follows its links, reading each page in a real browser. It respects robots.txt and stays on the start page's host unless you turn sameHost off. It waits up to waitSeconds for the crawl and returns the pages read so far. If the crawl is still running, web_crawl_get gives the rest. It does not use or create a session. Public pages only.

Inputs

Name	Type	Default	Allowed values	Description
`url`required	`string`	–	–	The page to start from.
`maxPages`	`integer`	20	–	How many pages to read, 1 up to your plan's limit: 200 on Free, 1,000 on paid plans.
`maxDepth`	`integer`	3	–	How many links away from the start page to go, 0 to 10.
`include`	`array of strings`	–	–	Regular expressions. A link is followed only if its URL matches one. At most 20.
`exclude`	`array of strings`	–	–	Regular expressions. A link whose URL matches one is skipped. At most 20.
`sameHost`	`boolean`	true	–	Stay on the start page's host name.
`format`	`string`	markdown	`markdown`, `text`, `html`	The format of each page's content.
`waitSeconds`	`integer`	45	–	How long to wait for the crawl before answering, 0 to 120. With 0 it answers right away.

Example call

This is what an MCP client sends. You do not write it yourself: ask the model in plain words and it calls the tool.

tools/call

{  "name": "web_crawl_start",  "arguments": {    "url": "https://example.com/docs",    "maxPages": 30,    "include": [      "/docs/"    ]  }}

What it returns

A header line with the crawl's id, its status and how many pages were read, failed and skipped by robots.txt. Then up to 10 pages, each with its title, address, HTTP status, depth and content. Then a line that says what to do next.

Result

Crawl crawl_2Gh7Kx0Lq9Zp4Nm3 of https://example.com/docs: completed; 3 pages read, 0 failed, 1 skipped by robots.txt.## Docs homehttps://example.com/docs (HTTP 200, depth 0)# Docs home…## Installhttps://example.com/docs/install (HTTP 200, depth 1)…That is every page.

When a call fails, the result is marked as an error. Its text starts with the API’s error code, such as unauthorized or session_not_running, followed by its message.

Annotations

The server tells every client what this tool does to the world, so a client can decide what to ask you about before it runs.

Annotation	Value
`title`	Crawl a website
`readOnlyHint`	`false`
`destructiveHint`	`false`
`openWorldHint`	`true`

Good to know

A call shows 10 pages at a time, and each page's content is cut at 3,500 characters. When there are more, the last line gives the after to pass to web_crawl_get.
A page that shows a CAPTCHA is not read, and the result says so.
The crawl always obeys robots.txt: pages it disallows are skipped and counted.
Your plan must include crawls, and a project can start 10 crawls a minute.
Crawl only sites you are allowed to crawl.
It starts a crawl job that uses your plan and credit, so it is not marked read-only: a client may ask before it runs. web_crawl_get, which only reads the result, is read-only.

The same thing with the API

Endpoint	Reference
`POST /v1/crawls`	Start a crawl
`GET /v1/crawls/{id}`	Get a crawl

These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.