Mapterhorn (mapterhorn.com, Protomaps-affiliated) publishes pre-built
Terrarium-encoded terrain as PMTiles: a global planet.pmtiles (z0-12)
plus ~450 regional chunks (z13+). Same pmtiles extract --bbox=...
mechanism offline.pmtiles already uses, just against their files —
confirmed live against their real manifest and the documented
Interlaken example. Replaces fetching every individual tile ourselves
(even after this session's earlier concurrency fix, still tens of
minutes for a real region) with 1-3 extract calls plus a merge when
the bbox spans more than one chunk: seconds, not minutes.
Removes CONCURRENT_TILE_FETCHES, write_terrarium_mbtiles, tile_range,
lng/lat_to_tile_x/y, fetch_terrarium_tile, and the sqlite3 gem
entirely — none of it is needed once the tiles arrive pre-packaged.
dem.js's point-elevation lookup (shadow.js's terrain-shadow overlay)
hardcoded a 256px PNG tile assumption that Mapterhorn's 512px WebP
tiles would have silently broken — now reads the actually-decoded
tile's own size and mime type instead of assuming either.
A logged-in user can now trigger, from a new "Offline data" sidebar
panel, what was previously a fully manual go-pmtiles workflow
(docs/development.adoc): regenerate either file for the current map
viewport, no area-picker (same "visible area is the area" convention
the Overpass POI plugin already uses).
- lib/maptool/pmtiles_builder.rb: two independent background jobs
(Thread + Mutex-guarded status Hash, same shape as app.rb's own GNSS
poller), each writing to a .tmp path and atomically renaming into
place so a build never serves a partially-written file mid-run.
offline.pmtiles is a single `pmtiles extract` against Protomaps'
public daily basemap build (byte-range extraction, not a full
~120GB download). terrain.pmtiles has no equivalent public
Terrarium-encoded archive to extract from, so it fetches individual
Terrarium tiles itself, packs them into an intermediate MBTiles
(new sqlite3 gem), then the same `pmtiles convert`.
- 4 new app.rb routes (POST .../build, GET .../build-status) plus
size/mtime added to the existing availability endpoints.
- public/js/offline_data.js: new sidebar panel, polling build-status
the same self-rescheduling-setTimeout shape livetrack.js already
uses for its own refresh loop.
- Both network-heavy pieces (the Protomaps build index, per-tile
fetches) are injectable, so test/test_pmtiles_builder.rb exercises
the actual build pipeline against a fake `pmtiles` binary without
hitting a real network, matching this suite's existing
keep-it-offline convention for external services.
data/plugins/p4n2gpx/ (outside git, see .gitignore) now holds a
plugin.json + run.rb wrapper around ../p4n2gpx/ruby/p4n2gpx.rb, so it
runs through the same --south/--west/--north/--east-in,
GeoJSON-on-stdout-out contract as the built-in overpass-pois plugin.
run.rb converts the requested bbox to the center+radius p4n2gpx.rb
itself expects, then reuses lib/maptool/gpx.rb to turn its .gpx output
into GeoJSON.
p4n2gpx.rb needs faraday-cookie_jar, which wasn't in the Gemfile yet
(faraday itself already was, transitively via oauth2) — added and
locked. Also wires P4N_USER/P4N_PASSWORD through docker-compose.yaml
and the deploy workflow's "Write .env" step, the same optional-secret
pattern as LIVETRACK_WEBHOOK_SECRET, since the container has no
browser for p4n2gpx.rb's Chrome-cookie auth fallback.
`bundle exec rake test` (minitest, a default gem but pinned in
Gemfile for reproducibility) now covers what ad-hoc scripts kept
re-verifying by hand during development: Gpx/Fit/TrackStats parsing,
GpxStore/LiveTrackStore file/TTL logic, LiveTrackClient's payload
parsing, and app.rb's routes end to end via Rack::MockRequest.
MAPTOOL_DATA_DIR (new, app.rb) lets the suite point GPX_STORE/
LIVETRACK_STORE at a throwaway directory instead of real data/ —
without it every run would leave uploaded test files behind. Each
lib file's old `if __FILE__ == $0` self-check is now redundant with
real tests covering the same ground, so removed rather than kept as
a second copy of the same assertions. Wired into ci.yml so it isn't
just a local convenience.
Maptool::Fit reads Garmin's binary FIT format directly rather than
adding a gem — the one Ruby option depends on activesupport, unmaintained
since 2022, for a single-purpose binary parser. Only "record" messages
are read (position + altitude; everything else is skipped by byte
length, never interpreted); field numbers and scale/offset are
Garmin's own (fit-python-sdk's profile.py), and the framing plus all
extracted values were checked byte-for-byte against 5 real device
files from python-fitparse's test suite before this shipped.
A .zip upload (Garmin's bulk/FIT export format) is expanded into its
.gpx/.fit entries via rubyzip — that one's an actual dependency, since
ZIP's own format isn't the kind of narrow, single-purpose parsing FIT
above is.
Upload/delete/move/create-folder now require logging in via Gitea
OAuth2 (only MAPTOOL_ALLOWED_GITEA_USER is granted write access);
reading stays public. Deploy secrets (session secret, OAuth2 client
credentials, allowed user) are written to .env from Gitea Actions
repo secrets on every deploy instead of living on the server by hand.
Also switch the container's bind address to "[::]" (dual-stack) so
the app is reachable over IPv6, not just IPv4.
Upload GPX files into a folder hierarchy (traversal-safe path
resolution against the store root), browse them in a tree panel, and
toggle each one's waypoints/tracks on the map. Waypoints render as
clickable points showing name, description and time; bare URLs in the
description are turned into safe links. GPX layers survive switching
between the Online/Offline base map, same as the radar overlay.
Adds rexml as an explicit dependency — it moved to a bundled gem in
this Ruby version and isn't picked up by Bundler otherwise.
randy