Local businesses
List the businesses of one kind in a place (cafés in Bath, bike shops in Utrecht, printers in Lyon) from OpenStreetMap's open data, then read the websites they list to say what each offers. For finding a place to go, a supplier or a venue.
When to use it: You are looking for a place, a supplier or a venue of one kind in a town.
| Uses | a shell session (osm.py: Nominatim and Overpass), extract over the businesses' websites |
|---|---|
| Needs | shell sessions and a model on the API (no keys of your own) |
| Site | OpenStreetMap's official services and the businesses' own websites; the runner uses stand-ins |
| Output | output/businesses.csv, output/result.json |
| Languages | Node and Python |
OpenStreetMap's services are free and shared: the script identifies itself, makes one request to each, and keeps queries small, as their usage policies ask. For bigger or repeated jobs, run your own Overpass instance or use an OSM data extract. The data is © OpenStreetMap contributors under the ODbL: the attribution is in every output, keep it when you share the list. KIND is an OSM tag such as amenity=cafe, shop=bicycle or craft=printer.
The code
The example as it is in its folder. Both versions do the same job and write the same output/, so one result check covers both.
/** * Local businesses: list the businesses of one kind in a place (cafés in Bath, bike shops in Utrecht…) from * OpenStreetMap's open data, then read the websites they list to say what each offers. A list for finding a place, * a supplier or a venue, with OSM's attribution. * * npx tsx node/index.ts (BOXLINE_API_KEY; PLACE, KIND (an OSM tag), LIMIT, ENRICH; a plan with shell sessions) * * osm.py queries OpenStreetMap from the session's shell (Nominatim and Overpass, within their usage policies); one * extract reads the businesses' own websites. Writes output/businesses.csv, output/result.json. */import { mkdirSync, readFileSync, writeFileSync } from "node:fs";import { dirname, join } from "node:path";import { fileURLToPath } from "node:url";import { Boxline } from "@boxline/sdk";const here = dirname(fileURLToPath(import.meta.url));const out = process.env.OUTPUT_DIR ?? "output";const env = { PLACE: process.env.PLACE ?? "Bath, United Kingdom", KIND: process.env.KIND ?? "amenity=cafe", LIMIT: process.env.LIMIT ?? "20", NOMINATIM_URL: process.env.NOMINATIM_URL ?? "https://nominatim.openstreetmap.org/search", OVERPASS_URL: process.env.OVERPASS_URL ?? "https://overpass-api.de/api/interpreter",};const enrich = Math.min(Number(process.env.ENRICH ?? 5), 10);const bx = new Boxline();type Business = { osm: string; name: string; address: string | null; lat: number; lon: number; website: string | null; phone: string | null; openingHours: string | null };const session = await bx.sessions.create({ browser: false, shell: true, timeout: 300, idleTimeout: 120, userMetadata: { example: "local-businesses" } });console.log(`Session: ${session.id}`);let osm: { place: string; bbox: number[]; kind: string; businesses: Business[]; attribution: string };try { await session.files.write("osm.py", readFileSync(join(here, "../osm.py"))); const r = await session.exec("python osm.py", { env, timeoutMs: 120_000 }); if (r.exitCode !== 0) throw new Error(`osm.py failed: ${(r.stderr || r.stdout).trim()}`); console.log(r.stdout.trim()); osm = JSON.parse(await session.files.readText("businesses.json"));} finally { await session.stop();}// What each offers, from its own website (the first ENRICH that list one).const sites = osm.businesses.filter((b) => b.website && /^https?:\/\//.test(b.website)).slice(0, enrich);let modelUsd = 0;const about = new Map<string, { description: string; specialities: string[] }>();if (sites.length) { const r = await bx.extract<{ sites: { url: string; description: string; specialities: string[] }[] }>({ urls: sites.map((b) => b.website!), prompt: "Each page is a local business's website. description: what it offers, one plain sentence. specialities: up to 3 things it is known for, a few words each. url: the page's address as given.", schema: { type: "object", properties: { sites: { type: "array", items: { type: "object", properties: { url: { type: "string" }, description: { type: "string" }, specialities: { type: "array", items: { type: "string" } } }, required: ["url", "description", "specialities"] } } }, required: ["sites"] }, }); modelUsd = r.usage.costUsd; r.data.sites.forEach((s, i) => about.set(sites.find((b) => b.website === s.url)?.website ?? sites[i]!.website!, { description: s.description, specialities: s.specialities }));}const businesses = osm.businesses.map((b) => ({ ...b, ...(b.website && about.has(b.website) ? about.get(b.website)! : { description: null, specialities: [] as string[] }) }));for (const b of businesses) console.log(` ${b.name}${b.address ? `, ${b.address}` : ""}${b.website ? ` ${b.website}` : ""}${b.description ? `\n ${b.description}` : ""}`);console.log(`\n${osm.attribution}`);const cols = ["name", "address", "website", "phone", "openingHours", "description", "lat", "lon", "osm"] as const;const cell = (v: unknown) => (v === null || v === undefined ? "" : /[",\n]/.test(String(v)) ? `"${String(v).replace(/"/g, '""')}"` : String(v));mkdirSync(out, { recursive: true });writeFileSync(join(out, "businesses.csv"), [cols.join(","), ...businesses.map((b) => cols.map((c) => cell(b[c])).join(",")), "", `# ${osm.attribution}`].join("\n") + "\n");writeFileSync(join(out, "result.json"), JSON.stringify({ place: osm.place, bbox: osm.bbox, kind: osm.kind, businesses, attribution: osm.attribution, usage: { modelUsd } }, null, 2));"""Local businesses: list the businesses of one kind in a place (cafés in Bath, bike shops in Utrecht…) fromOpenStreetMap's open data, then read the websites they list to say what each offers. A list for finding a place,a supplier or a venue, with OSM's attribution. python python/main.py (BOXLINE_API_KEY; PLACE, KIND (an OSM tag), LIMIT, ENRICH; a plan with shell sessions)osm.py queries OpenStreetMap from the session's shell (Nominatim and Overpass, within their usage policies); oneextract reads the businesses' own websites. Writes output/businesses.csv, output/result.json."""import csvimport jsonimport osimport refrom pathlib import Pathfrom boxline import Boxlinehere = Path(__file__).resolve().parentout = Path(os.environ.get("OUTPUT_DIR", "output"))env = { "PLACE": os.environ.get("PLACE", "Bath, United Kingdom"), "KIND": os.environ.get("KIND", "amenity=cafe"), "LIMIT": os.environ.get("LIMIT", "20"), "NOMINATIM_URL": os.environ.get("NOMINATIM_URL", "https://nominatim.openstreetmap.org/search"), "OVERPASS_URL": os.environ.get("OVERPASS_URL", "https://overpass-api.de/api/interpreter"),}enrich = min(int(os.environ.get("ENRICH", "5")), 10)bx = Boxline()with bx.sessions.create(browser=False, shell=True, timeout=300, idle_timeout=120, user_metadata={"example": "local-businesses"}) as session: print(f"Session: {session.id}", flush=True) session.files.write("osm.py", (here.parent / "osm.py").read_bytes()) r = session.exec("python osm.py", env=env, timeout_ms=120_000) if r["exitCode"] != 0: raise SystemExit(f"osm.py failed: {(r['stderr'] or r['stdout']).strip()}") print(r["stdout"].strip()) osm = json.loads(session.files.read_text("businesses.json"))# What each offers, from its own website (the first ENRICH that list one).sites = [b for b in osm["businesses"] if b["website"] and re.match(r"^https?://", b["website"])][:enrich]model_usd, about = 0.0, {}if sites: r = bx.extract( urls=[b["website"] for b in sites], prompt="Each page is a local business's website. description: what it offers, one plain sentence. specialities: up to 3 things it is known for, a few words each. url: the page's address as given.", schema={"type": "object", "properties": {"sites": {"type": "array", "items": {"type": "object", "properties": {"url": {"type": "string"}, "description": {"type": "string"}, "specialities": {"type": "array", "items": {"type": "string"}}}, "required": ["url", "description", "specialities"]}}}, "required": ["sites"]}, ) model_usd = r["usage"]["costUsd"] for i, s in enumerate(r["data"]["sites"]): url = next((b["website"] for b in sites if b["website"] == s["url"]), sites[i]["website"] if i < len(sites) else s["url"]) about[url] = {"description": s["description"], "specialities": s["specialities"]}businesses = [{**b, **about.get(b["website"] or "", {"description": None, "specialities": []})} for b in osm["businesses"]]for b in businesses: print(f" {b['name']}" + (f", {b['address']}" if b["address"] else "") + (f" {b['website']}" if b["website"] else "") + (f"\n {b['description']}" if b["description"] else ""))print(f"\n{osm['attribution']}")out.mkdir(parents=True, exist_ok=True)cols = ["name", "address", "website", "phone", "openingHours", "description", "lat", "lon", "osm"]with open(out / "businesses.csv", "w", newline="") as f: w = csv.writer(f) w.writerow(cols) for b in businesses: w.writerow(["" if b[c] is None else b[c] for c in cols]) f.write(f"\n# {osm['attribution']}\n")(out / "result.json").write_text(json.dumps({"place": osm["place"], "bbox": osm["bbox"], "kind": osm["kind"], "businesses": businesses, "attribution": osm["attribution"], "usage": {"modelUsd": model_usd}}, indent=2, ensure_ascii=False))"""Runs in the session's shell: finds the businesses of one kind in a place from OpenStreetMap, through its officialservices (Nominatim to find the place, Overpass to query it), following their usage policies: an identifyingUser-Agent, one request at a time, small queries. Inputs: PLACE, KIND (an OSM tag such as amenity=cafe), LIMIT,NOMINATIM_URL and OVERPASS_URL. Writes /workspace/businesses.json."""import jsonimport osimport timeimport urllib.errorimport urllib.parseimport urllib.requestUA = {"user-agent": "local-businesses (a Boxline example; https://boxline.dev)"}place = os.environ.get("PLACE", "Bath, United Kingdom")key, _, value = os.environ.get("KIND", "amenity=cafe").partition("=")limit = int(os.environ.get("LIMIT", "20"))nominatim = os.environ.get("NOMINATIM_URL", "https://nominatim.openstreetmap.org/search")overpass = os.environ.get("OVERPASS_URL", "https://overpass-api.de/api/interpreter")def get(url, data=None): """One request; the public servers are shared, so a busy answer (429, 5xx) or a timeout is retried after a pause.""" for attempt, pause in enumerate([5, 15, 0]): try: with urllib.request.urlopen(urllib.request.Request(url, data=data, headers=UA), timeout=60) as r: return json.load(r) except urllib.error.HTTPError as e: if e.code not in (429, 502, 503, 504) or not pause: raise SystemExit(f"{url} answered {e.code}: {e.read()[:200].decode('utf-8', 'replace')}") except (urllib.error.URLError, TimeoutError) as e: if not pause: raise SystemExit(f"{url} did not answer: {e}") print(f" {url} is busy; trying again in {pause} s") time.sleep(pause)# 1. Where the place is: its bounding box.hits = get(f"{nominatim}?{urllib.parse.urlencode({'q': place, 'format': 'json', 'limit': 1})}")if not hits: raise SystemExit(f"OpenStreetMap does not know {place!r}")south, north, west, east = (float(x) for x in hits[0]["boundingbox"])time.sleep(1) # Nominatim's policy: at most one request per second# 2. The businesses of that kind inside it (nodes, ways and relations; a way's centre as its location).query = f'[out:json][timeout:25];nwr["{key}"="{value}"]["name"]({south},{west},{north},{east});out center tags {limit * 2};'data = get(overpass, urllib.parse.urlencode({"data": query}).encode())found = []for el in data.get("elements", []): t = el.get("tags", {}) address = " ".join(x for x in [t.get("addr:housenumber"), t.get("addr:street")] if x) found.append({ "osm": f"https://www.openstreetmap.org/{el['type']}/{el['id']}", "name": t.get("name"), "address": ", ".join(x for x in [address, t.get("addr:postcode"), t.get("addr:city")] if x) or None, "lat": el.get("lat") or el.get("center", {}).get("lat"), "lon": el.get("lon") or el.get("center", {}).get("lon"), "website": t.get("website") or t.get("contact:website"), "phone": t.get("phone") or t.get("contact:phone"), "openingHours": t.get("opening_hours"), })found = sorted(found, key=lambda b: (b["website"] is None, b["name"] or ""))[:limit]json.dump({"place": hits[0].get("display_name", place), "bbox": [south, north, west, east], "kind": f"{key}={value}", "businesses": found, "attribution": "Data © OpenStreetMap contributors, ODbL 1.0 (https://www.openstreetmap.org/copyright)"}, open("/workspace/businesses.json", "w"), indent=1)print(f"{len(found)} {key}={value} in {hits[0].get('display_name', place)}")Inputs
Environment variables the example reads:
PLACE,KIND,LIMIT(default 20),ENRICH(websites to read, default 5).
Run it
Get the examples with git clone https://github.com/Boxline-dev/examples.git and open this one: cd examples/local-businesses. Put your API key in BOXLINE_API_KEY (and BOXLINE_API_URL for another API), then run either version from the example’s folder. Both write to output/ and stop their sessions when they finish.
# Node 18 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keynpm install @boxline/sdk tsxnpx tsx node/index.ts# Python 3.9 or newer, in the example's folderexport BOXLINE_API_KEY=… # your API keypip install boxline-sdkpython python/main.pyThen check the result, from the same folder: npx tsx check.ts output. It exits with 0 only when the result is right.
What its check proves
check.ts reads what the example wrote and passes only when the result is right, not merely when the example ran.
Every business has a name, lies inside the place and links to its OSM entry, and the attribution is kept. On the runner's stand-ins: 3 cafés with their addresses from OSM, Rosie's Kitchen (sourdough) and The Beanery (pour-over) described from their own websites, and Corner Cup, which lists none, left undescribed.
Source
github.com/Boxline-dev/examples/local-businesses on GitHub, with its README.md, check.ts and .env.example. The examples are open source. See all examples.
These docs describe a beta API. Endpoints, fields and SDK methods may change before general availability; breaking changes will be listed in the changelog.