Download all images
The browser finds every image on a page, the shell downloads them into the workspace with the browser's cookies (so images behind a sign-in work too) and zips them, and the files API brings the zip back.
When to use it: You need every image of a page as files, including images behind a sign-in.
| Uses | Playwright over CDP, exportCookies, files.write, exec (curl, zip), files.read |
|---|---|
| Needs | a plan with shell sessions; no model |
| Site | your page; the default is books.toscrape.com. The runner uses its stand-in gallery of three images |
| Output | output/images.zip and output/result.json |
| Languages | Node and Python |
download.sh is the shell script both versions upload and run in the session.
The code
The example as it is in its folder. Both versions do the same job and write the same output/, so one result check covers both.
/** * Download all images: the browser finds every image on a page, the shell downloads them into the workspace with the * browser's cookies and zips them, and the files API brings the zip back. * * npx tsx node/index.ts (BOXLINE_API_KEY; PAGE_URL for another page; a plan with shell sessions) * * Writes output/images.zip and output/result.json. */import { mkdirSync, readFileSync, writeFileSync } from "node:fs";import { join } from "node:path";import { Boxline } from "@boxline/sdk";import { chromium } from "playwright-core";const out = process.env.OUTPUT_DIR ?? "output";const url = process.env.PAGE_URL ?? "https://books.toscrape.com/";const bx = new Boxline();const session = await bx.sessions.create({ shell: true, timeout: 600, userMetadata: { example: "download-all-images" } });console.log(`Session: ${session.id}`);const browser = await chromium.connectOverCDP(session.connectUrl!);try { // 1. The browser: every image the page shows, as absolute addresses. const page = browser.contexts()[0]!.pages()[0]!; await page.goto(url, { waitUntil: "load" }); const srcs = await page.$$eval("img", (imgs) => imgs.map((i) => (i as HTMLImageElement).currentSrc || (i as HTMLImageElement).src)); const images = [...new Set(srcs)].filter((s) => /^https?:\/\//.test(s)).slice(0, 100); console.log(`${images.length} images on ${page.url()}`); // 2. The shell: download them with the browser's cookies (so images behind a sign-in work too), then zip them. await session.exportCookies("cookies.txt"); await session.files.write("images.txt", images.join("\n") + "\n"); await session.files.write("download.sh", readFileSync(new URL("../download.sh", import.meta.url))); const r = await session.exec("bash download.sh", { timeoutMs: 300_000 }); if (r.exitCode !== 0) throw new Error(`the download failed: ${r.stderr.trim()}`); if (r.stderr.trim()) console.log(r.stderr.trim()); const saved = Number(r.stdout.trim()); // 3. The files API: the zip, back on this computer. const zip = await session.files.read("output/images.zip"); mkdirSync(out, { recursive: true }); writeFileSync(join(out, "images.zip"), zip); console.log(`${saved} images saved in the workspace's output/images; output/images.zip (${Math.round(zip.length / 1024)} KB) is here`); writeFileSync(join(out, "result.json"), JSON.stringify({ page: url, found: images.length, saved, zipBytes: zip.length, images }, null, 2));} finally { await session.stop(); await browser.close().catch(() => undefined);}"""Download all images: the browser finds every image on a page, the shell downloads them into the workspace withthe browser's cookies and zips them, and the files API brings the zip back. python python/main.py (BOXLINE_API_KEY; PAGE_URL for another page; a plan with shell sessions)Writes output/images.zip and output/result.json."""import jsonimport osfrom pathlib import Pathfrom playwright.sync_api import sync_playwrightfrom boxline import Boxlineout = Path(os.environ.get("OUTPUT_DIR", "output"))url = os.environ.get("PAGE_URL", "https://books.toscrape.com/")script = (Path(__file__).parent.parent / "download.sh").read_bytes()bx = Boxline()with bx.sessions.create(shell=True, timeout=600, user_metadata={"example": "download-all-images"}) as session: print(f"Session: {session.id}", flush=True) with sync_playwright() as p: # 1. The browser: every image the page shows, as absolute addresses. page = p.chromium.connect_over_cdp(session.connect_url).contexts[0].pages[0] page.goto(url, wait_until="load") srcs = page.eval_on_selector_all("img", "imgs => imgs.map((i) => i.currentSrc || i.src)") images = [s for s in dict.fromkeys(srcs) if s.startswith(("http://", "https://"))][:100] print(f"{len(images)} images on {page.url}") # 2. The shell: download them with the browser's cookies (so images behind a sign-in work too), then zip them. session.export_cookies("cookies.txt") session.files.write("images.txt", "\n".join(images) + "\n") session.files.write("download.sh", script) r = session.exec("bash download.sh", timeout_ms=300_000) if r["exitCode"] != 0: raise SystemExit(f"the download failed: {r['stderr'].strip()}") if r["stderr"].strip(): print(r["stderr"].strip()) saved = int(r["stdout"].strip()) # 3. The files API: the zip, back on this computer. zip_bytes = session.files.read("output/images.zip") out.mkdir(parents=True, exist_ok=True) (out / "images.zip").write_bytes(zip_bytes) print(f"{saved} images saved in the workspace's output/images; output/images.zip ({len(zip_bytes) // 1024} KB) is here") result = {"page": url, "found": len(images), "saved": saved, "zipBytes": len(zip_bytes), "images": images} (out / "result.json").write_text(json.dumps(result, indent=2))#!/usr/bin/env bash# Runs in the session's shell: downloads every address in images.txt with the browser's cookies (cookies.txt, from# the session's cookie export) into output/images, then zips the folder. Prints how many images it saved.set -uo pipefailrm -rf output && mkdir -p output/imagesn=0while read -r url; do [ -z "$url" ] && continue n=$((n + 1)) path="${url%%\?*}" ext=$(printf '%s' "$path" | grep -oE '\.[A-Za-z0-9]{2,4}$' || true) curl -sSfL --max-time 30 -b cookies.txt -o "output/images/$(printf '%03d' "$n")${ext:-.img}" "$url" || echo "could not download $url" >&2done < images.txtcd output && zip -qr images.zip imagesls images | wc -lInputs
Environment variables the example reads:
PAGE_URL: the page whose images to save (defaulthttps://books.toscrape.com/).
Run it
Get the examples with git clone https://github.com/Boxline-dev/examples.git and open this one: cd examples/download-all-images. Put your API key in BOXLINE_API_KEY (and BOXLINE_API_URL for another API), then run either version from the example’s folder. Both write to output/ and stop their sessions when they finish.
# Node 18 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keynpm install @boxline/sdk playwright-core tsxnpx tsx node/index.ts# Python 3.9 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keypip install boxline-sdk playwrightpython python/main.pyThen check the result, from the same folder: npx tsx check.ts output. It exits with 0 only when the result is right.
What its check proves
check.ts reads what the example wrote and passes only when the result is right, not merely when the example ran.
The zip holds as many images as the page has; on the stand-in, the three PNGs byte for byte.
Source
github.com/Boxline-dev/examples/download-all-images on GitHub, with its README.md, check.ts and .env.example. The examples are open source. See all examples.
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.