Session replay as a GIF
Every session keeps a recording: one frame each time the screen changes, with the time and the page. This example browses a shop for a few seconds, then turns that session's recording into an MP4 and a GIF with ffmpeg in a second, shell-only session. Use it to show what an agent did in a pull request, a bug report or a demo.
When to use it: You want to show what an agent did in a pull request, a bug report or a demo, as a GIF made from the session's recording.
| Uses | sessions.create, session.click, session.back, sessions.recording, sessions.recordingFrame, a browser: false shell session, session.files, session.exec (make_replay.py: ffmpeg, ffprobe) |
|---|---|
| Needs | a plan with shell sessions |
| Site | [Books to Scrape](https://books.toscrape.com/), a demo shop made for scraping practice (START_URL for another copy of it) |
| Output | output/replay.mp4, output/replay.gif, the frames in output/frames/ and output/result.json |
| Languages | Node and Python |
How the clip is timed: each frame stays on screen until the next one was taken, at least 0.4 s so a quick change can be seen, at most 2 s so a long wait does not drag; the last frame stays 1.5 s. The MP4 is 1280 pixels wide (H.264, it plays everywhere); the GIF is 800 pixels wide at 5 frames a second with a palette made for the clip.
The recording works for any session, including an agent run: pass its session id to sessions.recording and the same make_replay.py makes its clip.
The code
The example as it is in its folder. Both versions do the same job and write the same output/, so one result check covers both.
/** * Session replay as a GIF: a short browse in one session, then that session's own recording (one JPEG frame per screen * change) turned into an MP4 and a GIF with ffmpeg in a second, shell-only session. The frames keep the timing they * had (long idle stretches shortened), so the clip plays back what happened. * * npx tsx node/index.ts (BOXLINE_API_KEY; START_URL; a plan with shell sessions) * * Writes output/replay.mp4, output/replay.gif, output/frames/ and output/result.json. */import { mkdirSync, readFileSync, writeFileSync } from "node:fs";import { dirname, join } from "node:path";import { fileURLToPath } from "node:url";import { Boxline } from "@boxline/sdk";const here = dirname(fileURLToPath(import.meta.url));const out = process.env.OUTPUT_DIR ?? "output";const start = process.env.START_URL ?? "https://books.toscrape.com/";const bx = new Boxline();// 1. The browse to record: the shop, a category, a book, and back.const browse = await bx.sessions.create({ timeout: 300, userMetadata: { example: "session-replay-gif" } });console.log(`Session: ${browse.id} (the browse)`);try { await browse.goto(start); await browse.click('a[href*="category/books/travel_2"]'); await browse.click("article.product_pod h3 a"); // A moment on the book's page, as a person would: the recording keeps a screen after a page loads, at most one per 250 ms. await browse.wait("article.product_page"); await browse.wait(800); await browse.back();} finally { await browse.stop(); // the recording keeps a last frame when the session stops}// 2. Its recording: the frames and when each was taken.const rec = await bx.sessions.recording(browse.id);if (rec.frames.length < 2) throw new Error(`the recording has ${rec.frames.length} frames`);mkdirSync(join(out, "frames"), { recursive: true });const t0 = Date.parse(rec.frames[0]!.at);const frames = [];for (const [i, f] of rec.frames.entries()) { const jpg = await bx.sessions.recordingFrame(browse.id, f.index); const file = `frames/${String(i).padStart(3, "0")}.jpg`; writeFileSync(join(out, file), jpg); frames.push({ file, t: (Date.parse(f.at) - t0) / 1000, url: f.url });}console.log(`${frames.length} frames over ${(frames.at(-1)!.t).toFixed(1)} s: ${[...new Set(frames.map((f) => f.url).filter(Boolean))].join(" → ")}`);// 3. The clip, made in a shell-only session with ffmpeg; the files go in and come back through the files API.const shell = await bx.sessions.create({ browser: false, shell: true, timeout: 600, userMetadata: { example: "session-replay-gif" } });console.log(`Session: ${shell.id} (ffmpeg)`);try { for (const f of frames) await shell.files.write(f.file, readFileSync(join(out, f.file))); await shell.files.write("frames.json", JSON.stringify(frames)); await shell.files.write("make_replay.py", readFileSync(join(here, "../make_replay.py"))); const r = await shell.exec("python3 make_replay.py", { timeoutMs: 300_000 }); if (r.exitCode !== 0) throw new Error(`make_replay.py failed: ${r.stderr.trim()}`); const made = JSON.parse(r.stdout); writeFileSync(join(out, "replay.mp4"), await shell.files.read("replay.mp4")); writeFileSync(join(out, "replay.gif"), await shell.files.read("replay.gif")); console.log(`replay.mp4: ${made.mp4.seconds} s, ${made.mp4.width}×${made.mp4.height}; replay.gif: ${made.gif.frames} frames`); writeFileSync( join(out, "result.json"), JSON.stringify({ start, sessionId: browse.id, recordingMs: rec.durationMs, ...made, frames }, null, 2), );} finally { await shell.stop();}"""Session replay as a GIF: a short browse in one session, then that session's own recording (one JPEG frame per screenchange) turned into an MP4 and a GIF with ffmpeg in a second, shell-only session. The frames keep the timing theyhad (long idle stretches shortened), so the clip plays back what happened. python python/main.py (BOXLINE_API_KEY; START_URL; a plan with shell sessions)Writes output/replay.mp4, output/replay.gif, output/frames/ and output/result.json."""import jsonimport osfrom datetime import datetimefrom pathlib import Pathfrom boxline import Boxlinehere = Path(__file__).resolve().parentout = Path(os.environ.get("OUTPUT_DIR", "output"))start = os.environ.get("START_URL", "https://books.toscrape.com/")bx = Boxline()# 1. The browse to record: the shop, a category, a book, and back.with bx.sessions.create(timeout=300, user_metadata={"example": "session-replay-gif"}) as browse: # stopped at the end: a last frame print(f"Session: {browse.id} (the browse)", flush=True) browse.goto(start) browse.click(selector='a[href*="category/books/travel_2"]') browse.click(selector="article.product_pod h3 a") # A moment on the book's page, as a person would: the recording keeps a screen after a page loads, at most one per 250 ms. browse.wait(selector="article.product_page") browse.wait(ms=800) browse.back()# 2. Its recording: the frames and when each was taken.rec = bx.sessions.recording(browse.id)if len(rec["frames"]) < 2: raise SystemExit(f"the recording has {len(rec['frames'])} frames")(out / "frames").mkdir(parents=True, exist_ok=True)def seconds(iso: str) -> float: return datetime.fromisoformat(iso.replace("Z", "+00:00")).timestamp()t0 = seconds(rec["frames"][0]["at"])frames = []for i, f in enumerate(rec["frames"]): name = f"frames/{i:03d}.jpg" (out / name).write_bytes(bx.sessions.recording_frame(browse.id, f["index"])) frames.append({"file": name, "t": seconds(f["at"]) - t0, "url": f.get("url")})pages = list(dict.fromkeys(f["url"] for f in frames if f["url"]))print(f"{len(frames)} frames over {frames[-1]['t']:.1f} s: {' → '.join(pages)}", flush=True)# 3. The clip, made in a shell-only session with ffmpeg; the files go in and come back through the files API.with bx.sessions.create(browser=False, shell=True, timeout=600, user_metadata={"example": "session-replay-gif"}) as shell: print(f"Session: {shell.id} (ffmpeg)", flush=True) for f in frames: shell.files.write(f["file"], (out / f["file"]).read_bytes()) shell.files.write("frames.json", json.dumps(frames)) shell.files.write("make_replay.py", (here.parent / "make_replay.py").read_bytes()) r = shell.exec("python3 make_replay.py", timeout_ms=300_000) if r["exitCode"] != 0: raise SystemExit(f"make_replay.py failed: {r['stderr'].strip()}") made = json.loads(r["stdout"]) (out / "replay.mp4").write_bytes(shell.files.read("replay.mp4")) (out / "replay.gif").write_bytes(shell.files.read("replay.gif"))print(f"replay.mp4: {made['mp4']['seconds']} s, {made['mp4']['width']}×{made['mp4']['height']}; replay.gif: {made['gif']['frames']} frames")result = {"start": start, "sessionId": browse.id, "recordingMs": rec["durationMs"], **made, "frames": frames}(out / "result.json").write_text(json.dumps(result, indent=2, ensure_ascii=False))"""Runs in the session's shell: frames/NNN.jpg and frames.json (each frame's time) to replay.mp4 and replay.gif withffmpeg. Each frame stays on screen as long as it did in the session, between MIN and MAX seconds (long idle stretchesare shortened). Prints a JSON summary measured with ffprobe."""import jsonimport subprocessMIN, MAX, LAST = 0.4, 2.0, 1.5frames = json.load(open("frames.json"))times = [f["t"] for f in frames]durations = [min(MAX, max(MIN, b - a)) for a, b in zip(times, times[1:])] + [LAST]with open("list.txt", "w") as f: for frame, d in zip(frames, durations): f.write(f"file '{frame['file']}'\nduration {d:.3f}\n") f.write(f"file '{frames[-1]['file']}'\n") # the concat demuxer needs the last file twice for its duration to countdef run(*args): subprocess.run(args, check=True, capture_output=True)run("ffmpeg", "-y", "-loglevel", "error", "-f", "concat", "-safe", "0", "-i", "list.txt", "-vf", "scale=1280:-2:flags=lanczos,fps=10,format=yuv420p", "-c:v", "libx264", "-movflags", "+faststart", "replay.mp4")run("ffmpeg", "-y", "-loglevel", "error", "-i", "replay.mp4", "-vf", "fps=5,scale=800:-1:flags=lanczos,split[a][b];[a]palettegen=max_colors=128[p];[b][p]paletteuse", "replay.gif")def probe(path): out = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "v:0", "-count_frames", "-show_entries", "stream=width,height,nb_read_frames:format=duration", "-of", "json", path], check=True, capture_output=True, text=True).stdout p = json.loads(out) s = p["streams"][0] return {"seconds": round(float(p["format"]["duration"]), 2), "width": s["width"], "height": s["height"], "frames": int(s["nb_read_frames"])}print(json.dumps({"screens": len(frames), "plannedSeconds": round(sum(durations), 2), "mp4": probe("replay.mp4"), "gif": probe("replay.gif")}))Inputs
Environment variables the example reads:
START_URL: the shop's start page (default: Books to Scrape). The browse opens its Travel category, the first book there, and goes back.
Run it
Get the examples with git clone https://github.com/Boxline-dev/examples.git and open this one: cd examples/session-replay-gif. Put your API key in BOXLINE_API_KEY (and BOXLINE_API_URL for another API), then run either version from the example’s folder. Both write to output/ and stop their sessions when they finish.
# Node 18 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keynpm install @boxline/sdk tsxnpx tsx node/index.ts# Python 3.9 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keypip install boxline-sdkpython python/main.pyThen check the result, from the same folder: npx tsx check.ts output. It exits with 0 only when the result is right.
What its check proves
check.ts reads what the example wrote and passes only when the result is right, not merely when the example ran.
npx tsx check.ts output exits with 0 only when the recording has at least three screens in time order, including the Travel category and a book's page, replay.mp4 is an MP4 whose length (read from its header) matches the frames' timing within half a second, and replay.gif is a GIF with a frame for every screen.
Source
github.com/Boxline-dev/examples/session-replay-gif on GitHub, with its README.md, check.ts and .env.example. The examples are open source. See all examples.
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.