4 Commits

Include renames

Author	SHA1	Message	Date
randy	f89dea5805	Build terrain.pmtiles from Mapterhorn instead of fetching tiles ourselves ... Mapterhorn (mapterhorn.com, Protomaps-affiliated) publishes pre-built Terrarium-encoded terrain as PMTiles: a global planet.pmtiles (z0-12) plus ~450 regional chunks (z13+). Same pmtiles extract --bbox=... mechanism offline.pmtiles already uses, just against their files — confirmed live against their real manifest and the documented Interlaken example. Replaces fetching every individual tile ourselves (even after this session's earlier concurrency fix, still tens of minutes for a real region) with 1-3 extract calls plus a merge when the bbox spans more than one chunk: seconds, not minutes. Removes CONCURRENT_TILE_FETCHES, write_terrarium_mbtiles, tile_range, lng/lat_to_tile_x/y, fetch_terrarium_tile, and the sqlite3 gem entirely — none of it is needed once the tiles arrive pre-packaged. dem.js's point-elevation lookup (shadow.js's terrain-shadow overlay) hardcoded a 256px PNG tile assumption that Mapterhorn's 512px WebP tiles would have silently broken — now reads the actually-decoded tile's own size and mime type instead of assuming either.	2026-09-11 12:26:03 +02:00
randy	d041c21270	Fetch terrain tiles concurrently with persistent connections ... Serial fetches (no connection reuse) measured ~650ms/tile, almost all TCP+TLS handshake — a real bbox at z0..15 is tens of thousands of tiles, so this was hours for a modest region. Now: 16 worker threads share a queue (I/O-bound, so the GVL doesn't get in the way), each keeping its own persistent connection across every tile it fetches. Measured live against the real Terrarium source: ~40ms/tile, ~16x faster. write_terrarium_mbtiles's SQLite writes and progress updates stay on one thread (draining a results Queue the workers feed), so nothing needed its own locking beyond the two Queues themselves.	2026-09-11 12:04:13 +02:00
randy	bbe9f38bda	Show build progress instead of a static "Building…" ... terrain.pmtiles can take minutes (tiles fetched one at a time) with no way to tell a slow build from a stuck one. Now reports tiles_done/ tiles_total into the same status Hash the frontend already polls, and the panel shows elapsed time (both files) plus tile counts (terrain only) so the numbers visibly move every ~3s poll instead of sitting on one static string.	2026-09-11 11:57:43 +02:00
randy	cd56ebe0a1	Add in-app regeneration of offline.pmtiles and terrain.pmtiles ... A logged-in user can now trigger, from a new "Offline data" sidebar panel, what was previously a fully manual go-pmtiles workflow (docs/development.adoc): regenerate either file for the current map viewport, no area-picker (same "visible area is the area" convention the Overpass POI plugin already uses). - lib/maptool/pmtiles_builder.rb: two independent background jobs (Thread + Mutex-guarded status Hash, same shape as app.rb's own GNSS poller), each writing to a .tmp path and atomically renaming into place so a build never serves a partially-written file mid-run. offline.pmtiles is a single `pmtiles extract` against Protomaps' public daily basemap build (byte-range extraction, not a full ~120GB download). terrain.pmtiles has no equivalent public Terrarium-encoded archive to extract from, so it fetches individual Terrarium tiles itself, packs them into an intermediate MBTiles (new sqlite3 gem), then the same `pmtiles convert`. - 4 new app.rb routes (POST .../build, GET .../build-status) plus size/mtime added to the existing availability endpoints. - public/js/offline_data.js: new sidebar panel, polling build-status the same self-rescheduling-setTimeout shape livetrack.js already uses for its own refresh loop. - Both network-heavy pieces (the Protomaps build index, per-tile fetches) are injectable, so test/test_pmtiles_builder.rb exercises the actual build pipeline against a fake `pmtiles` binary without hitting a real network, matching this suite's existing keep-it-offline convention for external services.	2026-09-11 01:25:08 +02:00