Replace offline basemap pipeline with Planetiler, unify online/offline styling
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 2m43s

Offline tiles were built from Protomaps' own basemap schema, needing a
completely separate hand-tuned theme (protomaps-themes-base + OFFLINE_THEME)
from the online Liberty style's ONLINE_STYLE_PATCH — the two could never be
made to match exactly no matter how much either side got tuned (today's
session found and fixed several mismatches: building opacity, park
boundaries, road casings, road widths).

Offline.pmtiles is now built with Planetiler's default OpenMapTiles profile
against Geofabrik per-country OSM extracts (osmium-merged when more than one
country is selected), the same vector schema Liberty already expects.
offlineStyle() now clones the same Liberty JSON (vendored as a local
snapshot for reliability with no internet) and applies the identical
ONLINE_STYLE_PATCH, just repointed at local PMTiles/glyphs/sprite — one
palette instead of two, correct on both sides by construction.

Also vendors Liberty's icon sprite for the first time (offline had none
before), adds Java/osmium-tool/planetiler.jar to the Docker image, and
removes the now-unused protomaps-themes-base dependency and the
theme-comparison debug page that found today's bugs.
This commit is contained in:
randy committed 2026-09-11 20:52:11 +02:00
1 parent 0874e8f8ed
commit d08ed41185
21 files changed
+4286 -512

No files matched your search

+43 -49
View File
@@ -23,40 +23,31 @@ explicit cache headers, so an edit to `public/` (or `views/`) can
appear not to have taken effect on the next reload. Force a real
reload (e.g. Ctrl/Cmd+Shift+R) if that happens.
`/theme_preview.html` is a standalone debug page (not linked from the
app itself) for comparing offline base-map themes side by side: a bare
`protomaps-themes-base` theme, an iframe of the real app's offline
style, and one of its online style, with pan/zoom kept in sync across
all three via the same URL-hash mechanism `map.js` already uses for
shareable links. It's a plain static file, so nothing gates it to
development — it's reachable on a production instance too, same as any
other file under `public/`. Harmless if it stays that way (it shows
nothing `/` doesn't already), but worth knowing if that's ever not the
case.
== Offline map data
To enable the offline style, place a PMTiles file at
`data/offline.pmtiles` (see `docs/architecture.adoc` for how it is
served and styled). Logged-in users can generate this from the app
itself — see "In-app regeneration" below — or it can be built by hand:
served and styled — the same OpenMapTiles schema/style the online map
uses, just from local data). Logged-in users can generate this from
the app itself — see "In-app regeneration" below — or it can be built
by hand:
* Extract a region from Protomaps' own daily full-planet basemap build
(https://build-metadata.protomaps.dev/builds.json[build index],
files at `https://build.protomaps.com/<key>`, ~120GB, ODbL/OSM
attribution) with the
https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI's `extract`
subcommand, which pulls only the byte ranges covering a bbox over
HTTP range requests rather than the whole file:
`pmtiles extract https://build.protomaps.com/<key> data/offline.pmtiles --bbox=west,south,east,north --maxzoom=15`.
(The old `app.protomaps.com` extract-builder UI this section used to
point at has since redirected elsewhere — this is the same
underlying mechanism, just the CLI instead of a web UI.)
* To convert some other vector tile source (e.g. an `.mbtiles` file)
into PMTiles instead, use the same CLI's `convert` subcommand:
`pmtiles convert in.mbtiles data/offline.pmtiles`. Its schema is
what `protomaps-themes-base` (the styling library used for the
offline style) expects.
* Download one or more country extracts from
https://download.geofabrik.de[Geofabrik] (`.osm.pbf`, ODbL/OSM
attribution) — e.g.
`curl -O https://download.geofabrik.de/europe/slovakia-latest.osm.pbf`.
* If more than one country, merge them with
https://github.com/osmcode/osmium-tool[osmium-tool]:
`osmium merge slovakia-latest.osm.pbf austria-latest.osm.pbf -o merged.osm.pbf`
(each Geofabrik file is already clipped to that country's real
border, so no separate bbox/polygon clip step is needed either
before or after — skip this step entirely for a single country).
* Tile it with https://github.com/onthegomap/planetiler[Planetiler]'s
default profile, which *is* OpenMapTiles — no `--profile` flag
needed — writing PMTiles directly:
`java -jar planetiler.jar --osm-path=merged.osm.pbf --output=data/offline.pmtiles`
(needs Java 21+; `planetiler.jar` from
https://github.com/onthegomap/planetiler/releases/latest[the latest release]).
Once the file is in place, `GET /api/offline-tiles` reports
`available: true` and the map defaults to the offline style.
@@ -113,14 +104,15 @@ just what changed — fine given extraction is fast (byte-range requests
against remote archives, not per-tile fetches), but worth revisiting if
someone selects half of Europe at once and finds it slow.
This wraps exactly the manual `go-pmtiles` workflows documented above
(`lib/maptool/pmtiles_builder.rb`) — the server needs the
https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI installed and
on `PATH` (override with the `PMTILES_BIN` env var if it lives
somewhere else), using `--region=<geojson>` (a `FeatureCollection` of
the selected countries — go-pmtiles treats multiple features as their
union already, no union/dissolve math needed on this side) rather than
`--bbox=` for precise, per-country extraction. terrain.pmtiles extracts
This wraps exactly the manual workflows documented above
(`lib/maptool/pmtiles_builder.rb`). offline.pmtiles downloads each
selected country's Geofabrik extract, `osmium merge`s them if there's
more than one, then tiles the result with Planetiler — the server
needs `osmium` and `java` + `planetiler.jar` installed and on `PATH`
(override with the `OSMIUM_BIN`/`JAVA_BIN`/`PLANETILER_JAR` env vars if
they live somewhere else; the Docker image installs all three, see
`Dockerfile`). terrain.pmtiles still uses `go-pmtiles`
(`PMTILES_BIN`) against Mapterhorn, unrelated to this — it extracts
`planet.pmtiles` (always) plus whichever Mapterhorn regional chunks
intersect the selection, `merge`s them if there was more than one —
always safe regardless of how many/which countries are selected, since
@@ -139,18 +131,20 @@ starting a second while one's running gets a 409, as does starting
either with nothing selected yet.
Each successful build also writes a small sidecar (`offline.pmtiles.
meta.json` / `terrain.pmtiles.meta.json` — the Protomaps build date used,
or the Mapterhorn source names/md5sums) recording what was actually
built, since the `.pmtiles` file itself carries no such record. `GET
/api/offline-tiles`/`/api/terrain-tiles` include it as `meta`, shown in
the panel next to the existing size/updated line. `GET .../check-update`
compares that recorded build against the live upstream index/manifest
(without rebuilding anything) and reports `update_available: true/false/
null` — `null` means "don't know" (no local build yet, or the upstream
check itself failed), kept as its own route rather than folded into the
plain status endpoints above since those are on the page-load hot path
(the topbar's offline-status dot) and this one reaches an external
service, only worth doing when the Offline data panel is actually open.
meta.json` — each source country's code/Geofabrik slug/Last-Modified;
`terrain.pmtiles.meta.json` — the Mapterhorn source names/md5sums)
recording what was actually built, since the `.pmtiles` file itself
carries no such record. `GET /api/offline-tiles`/`/api/terrain-tiles`
include it as `meta`, shown in the panel next to the existing
size/updated line. `GET .../check-update` compares that recorded build
against each source's live upstream state (a Geofabrik `HEAD` per
country for offline, the Mapterhorn manifest for terrain — without
rebuilding anything) and reports `update_available: true/false/null` —
`null` means "don't know" (no local build yet, or a check itself
failed), kept as its own route rather than folded into the plain
status endpoints above since those are on the page-load hot path (the
topbar's offline-status dot) and this one reaches an external service,
only worth doing when the Offline data panel is actually open.
== Camper-relevant POIs (Overpass)