Replace offline basemap pipeline with Planetiler, unify online/offline styling
Offline tiles were built from Protomaps' own basemap schema, needing a completely separate hand-tuned theme (protomaps-themes-base + OFFLINE_THEME) from the online Liberty style's ONLINE_STYLE_PATCH — the two could never be made to match exactly no matter how much either side got tuned (today's session found and fixed several mismatches: building opacity, park boundaries, road casings, road widths). Offline.pmtiles is now built with Planetiler's default OpenMapTiles profile against Geofabrik per-country OSM extracts (osmium-merged when more than one country is selected), the same vector schema Liberty already expects. offlineStyle() now clones the same Liberty JSON (vendored as a local snapshot for reliability with no internet) and applies the identical ONLINE_STYLE_PATCH, just repointed at local PMTiles/glyphs/sprite — one palette instead of two, correct on both sides by construction. Also vendors Liberty's icon sprite for the first time (offline had none before), adds Java/osmium-tool/planetiler.jar to the Docker image, and removes the now-unused protomaps-themes-base dependency and the theme-comparison debug page that found today's bugs.
This commit is contained in:
1 parent
0874e8f8ed
commit
d08ed41185
21 files changed
+4286
-512
No files matched your search
+43
-49
@@ -23,40 +23,31 @@ explicit cache headers, so an edit to `public/` (or `views/`) can
|
||||
appear not to have taken effect on the next reload. Force a real
|
||||
reload (e.g. Ctrl/Cmd+Shift+R) if that happens.
|
||||
|
||||
`/theme_preview.html` is a standalone debug page (not linked from the
|
||||
app itself) for comparing offline base-map themes side by side: a bare
|
||||
`protomaps-themes-base` theme, an iframe of the real app's offline
|
||||
style, and one of its online style, with pan/zoom kept in sync across
|
||||
all three via the same URL-hash mechanism `map.js` already uses for
|
||||
shareable links. It's a plain static file, so nothing gates it to
|
||||
development — it's reachable on a production instance too, same as any
|
||||
other file under `public/`. Harmless if it stays that way (it shows
|
||||
nothing `/` doesn't already), but worth knowing if that's ever not the
|
||||
case.
|
||||
|
||||
== Offline map data
|
||||
|
||||
To enable the offline style, place a PMTiles file at
|
||||
`data/offline.pmtiles` (see `docs/architecture.adoc` for how it is
|
||||
served and styled). Logged-in users can generate this from the app
|
||||
itself — see "In-app regeneration" below — or it can be built by hand:
|
||||
served and styled — the same OpenMapTiles schema/style the online map
|
||||
uses, just from local data). Logged-in users can generate this from
|
||||
the app itself — see "In-app regeneration" below — or it can be built
|
||||
by hand:
|
||||
|
||||
* Extract a region from Protomaps' own daily full-planet basemap build
|
||||
(https://build-metadata.protomaps.dev/builds.json[build index],
|
||||
files at `https://build.protomaps.com/<key>`, ~120GB, ODbL/OSM
|
||||
attribution) with the
|
||||
https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI's `extract`
|
||||
subcommand, which pulls only the byte ranges covering a bbox over
|
||||
HTTP range requests rather than the whole file:
|
||||
`pmtiles extract https://build.protomaps.com/<key> data/offline.pmtiles --bbox=west,south,east,north --maxzoom=15`.
|
||||
(The old `app.protomaps.com` extract-builder UI this section used to
|
||||
point at has since redirected elsewhere — this is the same
|
||||
underlying mechanism, just the CLI instead of a web UI.)
|
||||
* To convert some other vector tile source (e.g. an `.mbtiles` file)
|
||||
into PMTiles instead, use the same CLI's `convert` subcommand:
|
||||
`pmtiles convert in.mbtiles data/offline.pmtiles`. Its schema is
|
||||
what `protomaps-themes-base` (the styling library used for the
|
||||
offline style) expects.
|
||||
* Download one or more country extracts from
|
||||
https://download.geofabrik.de[Geofabrik] (`.osm.pbf`, ODbL/OSM
|
||||
attribution) — e.g.
|
||||
`curl -O https://download.geofabrik.de/europe/slovakia-latest.osm.pbf`.
|
||||
* If more than one country, merge them with
|
||||
https://github.com/osmcode/osmium-tool[osmium-tool]:
|
||||
`osmium merge slovakia-latest.osm.pbf austria-latest.osm.pbf -o merged.osm.pbf`
|
||||
(each Geofabrik file is already clipped to that country's real
|
||||
border, so no separate bbox/polygon clip step is needed either
|
||||
before or after — skip this step entirely for a single country).
|
||||
* Tile it with https://github.com/onthegomap/planetiler[Planetiler]'s
|
||||
default profile, which *is* OpenMapTiles — no `--profile` flag
|
||||
needed — writing PMTiles directly:
|
||||
`java -jar planetiler.jar --osm-path=merged.osm.pbf --output=data/offline.pmtiles`
|
||||
(needs Java 21+; `planetiler.jar` from
|
||||
https://github.com/onthegomap/planetiler/releases/latest[the latest release]).
|
||||
|
||||
Once the file is in place, `GET /api/offline-tiles` reports
|
||||
`available: true` and the map defaults to the offline style.
|
||||
@@ -113,14 +104,15 @@ just what changed — fine given extraction is fast (byte-range requests
|
||||
against remote archives, not per-tile fetches), but worth revisiting if
|
||||
someone selects half of Europe at once and finds it slow.
|
||||
|
||||
This wraps exactly the manual `go-pmtiles` workflows documented above
|
||||
(`lib/maptool/pmtiles_builder.rb`) — the server needs the
|
||||
https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI installed and
|
||||
on `PATH` (override with the `PMTILES_BIN` env var if it lives
|
||||
somewhere else), using `--region=<geojson>` (a `FeatureCollection` of
|
||||
the selected countries — go-pmtiles treats multiple features as their
|
||||
union already, no union/dissolve math needed on this side) rather than
|
||||
`--bbox=` for precise, per-country extraction. terrain.pmtiles extracts
|
||||
This wraps exactly the manual workflows documented above
|
||||
(`lib/maptool/pmtiles_builder.rb`). offline.pmtiles downloads each
|
||||
selected country's Geofabrik extract, `osmium merge`s them if there's
|
||||
more than one, then tiles the result with Planetiler — the server
|
||||
needs `osmium` and `java` + `planetiler.jar` installed and on `PATH`
|
||||
(override with the `OSMIUM_BIN`/`JAVA_BIN`/`PLANETILER_JAR` env vars if
|
||||
they live somewhere else; the Docker image installs all three, see
|
||||
`Dockerfile`). terrain.pmtiles still uses `go-pmtiles`
|
||||
(`PMTILES_BIN`) against Mapterhorn, unrelated to this — it extracts
|
||||
`planet.pmtiles` (always) plus whichever Mapterhorn regional chunks
|
||||
intersect the selection, `merge`s them if there was more than one —
|
||||
always safe regardless of how many/which countries are selected, since
|
||||
@@ -139,18 +131,20 @@ starting a second while one's running gets a 409, as does starting
|
||||
either with nothing selected yet.
|
||||
|
||||
Each successful build also writes a small sidecar (`offline.pmtiles.
|
||||
meta.json` / `terrain.pmtiles.meta.json` — the Protomaps build date used,
|
||||
or the Mapterhorn source names/md5sums) recording what was actually
|
||||
built, since the `.pmtiles` file itself carries no such record. `GET
|
||||
/api/offline-tiles`/`/api/terrain-tiles` include it as `meta`, shown in
|
||||
the panel next to the existing size/updated line. `GET .../check-update`
|
||||
compares that recorded build against the live upstream index/manifest
|
||||
(without rebuilding anything) and reports `update_available: true/false/
|
||||
null` — `null` means "don't know" (no local build yet, or the upstream
|
||||
check itself failed), kept as its own route rather than folded into the
|
||||
plain status endpoints above since those are on the page-load hot path
|
||||
(the topbar's offline-status dot) and this one reaches an external
|
||||
service, only worth doing when the Offline data panel is actually open.
|
||||
meta.json` — each source country's code/Geofabrik slug/Last-Modified;
|
||||
`terrain.pmtiles.meta.json` — the Mapterhorn source names/md5sums)
|
||||
recording what was actually built, since the `.pmtiles` file itself
|
||||
carries no such record. `GET /api/offline-tiles`/`/api/terrain-tiles`
|
||||
include it as `meta`, shown in the panel next to the existing
|
||||
size/updated line. `GET .../check-update` compares that recorded build
|
||||
against each source's live upstream state (a Geofabrik `HEAD` per
|
||||
country for offline, the Mapterhorn manifest for terrain — without
|
||||
rebuilding anything) and reports `update_available: true/false/null` —
|
||||
`null` means "don't know" (no local build yet, or a check itself
|
||||
failed), kept as its own route rather than folded into the plain
|
||||
status endpoints above since those are on the page-load hot path (the
|
||||
topbar's offline-status dot) and this one reaches an external service,
|
||||
only worth doing when the Offline data panel is actually open.
|
||||
|
||||
== Camper-relevant POIs (Overpass)
|
||||
|
||||
|
||||
Reference in new issue
Block a user