Form anatomy
Take a web form apart without submitting it: every field with its label, type and rules, the messages the browser itself shows for an empty and for a wrong value, the hidden fields (and whether one is a CSRF token), where and how it posts, and warnings. For testing and documenting your own forms.
When to use it: You want to test or document your own forms without submitting them.
| Uses | a session, evaluate (anatomy.js in the page), screenshot |
|---|---|
| Needs | any plan; no model |
| Site | your form; the default is httpbin.org's demo order form; the runner uses a stand-in sign-up form |
| Output | output/form.md (a table of fields and messages), output/form.png, output/result.json |
| Languages | Node and Python |
Nothing is ever submitted: the browser's constraint validation (checkValidity) gives each field's message for an empty value and for a value its type, pattern or range must refuse (not-an-email, one below min, !! for a pattern), and the field is put back as it was. A minimum length is reported from the attribute: browsers only check it for typed values. Warnings: a password without HTTPS, a password sent with GET, posting to another site, fields without labels, and a POST form without a CSRF token among its hidden fields.
The code
The example as it is in its folder. Both versions do the same job and write the same output/, so one result check covers both.
/** * Form anatomy: take a web form apart without submitting it. Every field with its label, type and rules (required, * length, range, pattern, autocomplete), the messages the browser itself shows for an empty and for a wrong value, * the hidden fields (and whether one is a CSRF token), where and how it posts, and warnings (passwords without HTTPS, * posting to another site, fields without labels). For testing and documenting your own forms. * * npx tsx node/index.ts (BOXLINE_API_KEY; FORM_URL, FORM_SELECTOR) * * anatomy.js runs in the page; no model is used and nothing is submitted. Writes output/result.json and output/form.md. */import { mkdirSync, readFileSync, writeFileSync } from "node:fs";import { dirname, join } from "node:path";import { fileURLToPath } from "node:url";import { Boxline } from "@boxline/sdk";const here = dirname(fileURLToPath(import.meta.url));const out = process.env.OUTPUT_DIR ?? "output";const url = process.env.FORM_URL ?? "https://httpbin.org/forms/post";const selector = process.env.FORM_SELECTOR ?? "";const anatomy = readFileSync(join(here, "../anatomy.js"), "utf8").replace(/^\s*\/\/.*$/gm, "").trim();const bx = new Boxline();type Field = { name: string; label: string; type: string; required: boolean; minlength: string | null; maxlength: string | null; min: string | null; max: string | null; pattern: string | null; title: string | null; autocomplete: string | null; emptyMessage: string | null; invalidExample: string | null; invalidMessage: string | null };type Anatomy = { page: string; action: string; method: string; enctype: string; target: string | null; novalidate: boolean; scriptedSubmit: boolean; fields: Field[]; hidden: { name: string; looksLikeToken: boolean }[]; csrfToken: boolean; warnings: string[] };const session = await bx.sessions.create({ timeout: 180, userMetadata: { example: "form-anatomy" } });console.log(`Session: ${session.id}`);let form: Anatomy | null;try { await session.goto(url, { waitUntil: "load" }); form = await session.evaluate<Anatomy | null>(`(${anatomy})(${JSON.stringify(selector)})`); const shot = await session.screenshot({ fullPage: true }); mkdirSync(out, { recursive: true }); writeFileSync(join(out, "form.png"), Buffer.from(shot.data, "base64"));} finally { await session.stop();}if (!form) throw new Error(`no form${selector ? ` matches ${selector}` : ""} on ${url}`);const rules = (f: Field) => [f.required && "required", f.minlength && `at least ${f.minlength} characters`, f.maxlength && `at most ${f.maxlength}`, f.min && `≥ ${f.min}`, f.max && `≤ ${f.max}`, f.pattern && `pattern ${f.pattern}`, f.autocomplete && `autocomplete=${f.autocomplete}`] .filter(Boolean) .join(", ");console.log(`${form.method.toUpperCase()} ${form.action} (${form.enctype})${form.scriptedSubmit ? ", submitted by a script" : ""}${form.novalidate ? ", browser checks off" : ""}`);for (const f of form.fields) console.log(` ${f.label || f.name} [${f.type}] ${rules(f)}${f.emptyMessage ? `\n empty: "${f.emptyMessage}"` : ""}${f.invalidMessage ? `\n "${f.invalidExample}": "${f.invalidMessage}"` : ""}`);console.log(`Hidden: ${form.hidden.map((h) => `${h.name}${h.looksLikeToken ? " (token)" : ""}`).join(", ") || "none"}`);for (const w of form.warnings) console.log(` ! ${w}`);const md = [ `# Form at ${form.page}`, "", `\`${form.method.toUpperCase()} ${form.action}\` (${form.enctype})${form.scriptedSubmit ? ", submitted by a script" : ""}${form.novalidate ? ", browser checks switched off" : ""}.`, "", "| Field | Type | Rules | Empty | A wrong value |", "|---|---|---|---|---|", ...form.fields.map((f) => `| ${f.label || f.name} (\`${f.name}\`) | ${f.type} | ${rules(f) || "none"} | ${f.emptyMessage ?? "accepted"} | ${f.invalidExample ? `\`${f.invalidExample}\`: ${f.invalidMessage ?? "accepted"}` : ""} |`), "", `Hidden fields: ${form.hidden.map((h) => `\`${h.name}\`${h.looksLikeToken ? " (a token)" : ""}`).join(", ") || "none"}.`, "", ...(form.warnings.length ? ["## Warnings", "", ...form.warnings.map((w) => `- ${w}`), ""] : []), "", "",].join("\n");writeFileSync(join(out, "form.md"), md);writeFileSync(join(out, "result.json"), JSON.stringify({ url, selector: selector || null, ...form }, null, 2));"""Form anatomy: take a web form apart without submitting it. Every field with its label, type and rules (required,length, range, pattern, autocomplete), the messages the browser itself shows for an empty and for a wrong value,the hidden fields (and whether one is a CSRF token), where and how it posts, and warnings (passwords without HTTPS,posting to another site, fields without labels). For testing and documenting your own forms. python python/main.py (BOXLINE_API_KEY; FORM_URL, FORM_SELECTOR)anatomy.js runs in the page; no model is used and nothing is submitted. Writes output/result.json and output/form.md."""import jsonimport osimport refrom pathlib import Pathfrom boxline import Boxlinehere = Path(__file__).resolve().parentout = Path(os.environ.get("OUTPUT_DIR", "output"))url = os.environ.get("FORM_URL", "https://httpbin.org/forms/post")selector = os.environ.get("FORM_SELECTOR", "")anatomy = re.sub(r"^\s*//.*$", "", (here.parent / "anatomy.js").read_text(), flags=re.M).strip()bx = Boxline()with bx.sessions.create(timeout=180, user_metadata={"example": "form-anatomy"}) as session: print(f"Session: {session.id}", flush=True) session.goto(url, wait_until="load") form = session.evaluate(f"({anatomy})({json.dumps(selector)})") out.mkdir(parents=True, exist_ok=True) (out / "form.png").write_bytes(session.screenshot(full_page=True))if not form: raise SystemExit(f"no form{' matches ' + selector if selector else ''} on {url}")def rules(f): parts = ["required" if f["required"] else None, f["minlength"] and f"at least {f['minlength']} characters", f["maxlength"] and f"at most {f['maxlength']}", f["min"] and f"≥ {f['min']}", f["max"] and f"≤ {f['max']}", f["pattern"] and f"pattern {f['pattern']}", f["autocomplete"] and f"autocomplete={f['autocomplete']}"] return ", ".join(p for p in parts if p)flags = (", submitted by a script" if form["scriptedSubmit"] else "") + (", browser checks off" if form["novalidate"] else "")print(f"{form['method'].upper()} {form['action']} ({form['enctype']}){flags}")for f in form["fields"]: line = f" {f['label'] or f['name']} [{f['type']}] {rules(f)}" if f["emptyMessage"]: line += f"\n empty: \"{f['emptyMessage']}\"" if f["invalidMessage"]: line += f"\n \"{f['invalidExample']}\": \"{f['invalidMessage']}\"" print(line)print("Hidden: " + (", ".join(f"{h['name']}{' (token)' if h['looksLikeToken'] else ''}" for h in form["hidden"]) or "none"))for w in form["warnings"]: print(f" ! {w}")md = [f"# Form at {form['page']}", "", f"`{form['method'].upper()} {form['action']}` ({form['enctype']}){flags}.", "", "| Field | Type | Rules | Empty | A wrong value |", "|---|---|---|---|---|"]for f in form["fields"]: wrong = f"`{f['invalidExample']}`: {f['invalidMessage'] or 'accepted'}" if f["invalidExample"] else "" md.append(f"| {f['label'] or f['name']} (`{f['name']}`) | {f['type']} | {rules(f) or 'none'} | {f['emptyMessage'] or 'accepted'} | {wrong} |")md += ["", "Hidden fields: " + (", ".join(f"`{h['name']}`" + (" (a token)" if h["looksLikeToken"] else "") for h in form["hidden"]) or "none") + ".", ""]if form["warnings"]: md += ["## Warnings", ""] + [f"- {w}" for w in form["warnings"]] + [""]md += ["", ""](out / "form.md").write_text("\n".join(md))(out / "result.json").write_text(json.dumps({"url": url, "selector": selector or None, **form}, indent=2, ensure_ascii=False))// A function of the form's CSS selector (empty: the form with the most fields). Runs in the page (session.evaluate)// and takes the form apart without submitting it: its fields and their rules, the browser's own validation messages// for an empty and for a wrong value (checkValidity, never submit), hidden fields, and where and how it posts.(selector) => { const form = selector ? document.querySelector(selector) : [...document.forms].sort((a, b) => b.elements.length - a.elements.length)[0]; if (!form) return null; const labelOf = (el) => (el.labels && el.labels[0] ? el.labels[0].textContent : el.getAttribute("aria-label") || el.placeholder || "").trim().replace(/\s+/g, " "); // A value each type should refuse (programmatic values are checked for type, pattern, min and max; not for minlength). const wrong = (el) => { if (el.type === "email") return "not-an-email"; if (el.type === "url") return "not a url"; if (el.type === "number" && el.min !== "") return String(Number(el.min) - 1); if (el.type === "number" && el.max !== "") return String(Number(el.max) + 1); if (el.pattern) return "!!"; return null; }; const fields = []; const hidden = []; for (const el of form.elements) { if (!el.name || ["submit", "button", "reset", "image"].includes(el.type)) continue; if (el.type === "hidden") { hidden.push({ name: el.name, looksLikeToken: /csrf|xsrf|token|authenticity|nonce/i.test(el.name) && (el.value || "").length >= 6 }); continue; } const kept = el.type === "checkbox" || el.type === "radio" ? el.checked : el.value; el.checkValidity(); const emptyMessage = el.validationMessage || null; const bad = wrong(el); let invalidMessage = null; if (bad !== null) { el.value = bad; el.checkValidity(); invalidMessage = el.validationMessage || null; } if (el.type === "checkbox" || el.type === "radio") el.checked = kept; else el.value = kept; const attr = (n) => el.getAttribute(n); fields.push({ name: el.name, label: labelOf(el), type: el.type, required: el.required, minlength: attr("minlength"), maxlength: attr("maxlength"), min: attr("min"), max: attr("max"), pattern: attr("pattern"), title: attr("title"), autocomplete: attr("autocomplete"), emptyMessage, invalidExample: bad, invalidMessage }); } const action = new URL(form.action, location.href); const warnings = []; const hasPassword = fields.some((f) => f.type === "password"); if (hasPassword && location.protocol !== "https:") warnings.push("a password field on a page without HTTPS"); if (hasPassword && action.protocol !== "https:") warnings.push("the password is posted without HTTPS"); if (action.host !== location.host) warnings.push(`it posts to another site: ${action.host}`); if ((form.getAttribute("method") || "get").toLowerCase() === "get" && hasPassword) warnings.push("a password sent with GET ends up in the address and logs"); for (const f of fields) if (!f.label) warnings.push(`the field ${f.name} has no label`); if (form.method === "post" && !hidden.some((h) => h.looksLikeToken)) warnings.push("no CSRF token among the hidden fields (the site may use a cookie or header instead)"); return { page: location.href, action: action.href, method: (form.getAttribute("method") || "get").toLowerCase(), enctype: form.enctype, target: form.target || null, novalidate: form.noValidate, scriptedSubmit: Boolean(form.getAttribute("onsubmit")), fields, hidden, csrfToken: hidden.some((h) => h.looksLikeToken), warnings, };}Inputs
Environment variables the example reads:
FORM_URL,FORM_SELECTOR(default: the form with the most fields).
Run it
Get the examples with git clone https://github.com/Boxline-dev/examples.git and open this one: cd examples/form-anatomy. Put your API key in BOXLINE_API_KEY (and BOXLINE_API_URL for another API), then run either version from the example’s folder. Both write to output/ and stop their sessions when they finish.
# Node 18 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keynpm install @boxline/sdk tsxnpx tsx node/index.ts# Python 3.9 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keypip install boxline-sdkpython python/main.pyThen check the result, from the same folder: npx tsx check.ts output. It exits with 0 only when the result is right.
What its check proves
check.ts reads what the example wrote and passes only when the result is right, not merely when the example ran.
A form was read and every required field has the browser's message for an empty value. On the runner's stand-in form: it posts with a CSRF token, full name, email and password are required, the browser refuses "not-an-email", age 17 and a bad postcode, the password needs 12 characters, and the stand-in site saw no submission.
Source
github.com/Boxline-dev/examples/form-anatomy on GitHub, with its README.md, check.ts and .env.example. The examples are open source. See all examples.
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.