Commit Graph
192 Commits
Author SHA1 Message Date
randy 33be95ef59 Send a User-Agent to Overpass, overpass-api.de 406s Ruby's default
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 13s
The Camper POIs plugin died with "plugin exited with status 1": Net::HTTP's
bare "Ruby" User-Agent now gets a 406 Not Acceptable from overpass-api.de,
any identifying one goes through.
2026-10-09 21:51:24 +02:00
randy ede21454a2 Draw offline hiking trails sharing a path as parallel lanes
CI / check (push) Successful in 22s
CI / deploy (push) Successful in 12s
Several marked routes running down one path drew as a single line, only
the topmost color visible. maplibre-gl-lanes puts each route in its own
lane; OSM relations already share exact vertices, which is all it needs.
One LaneLayer across every picked country; click hit-testing and the
highlight move to queryLane/setHighlight.
2026-10-09 21:46:05 +02:00
randy 9c80a20f3e compose: enable IPv6 on default network (fixes broken container DNS on brick)
CI / check (push) Successful in 38s
CI / deploy (push) Successful in 12s
2026-10-08 12:28:05 +02:00
randy 43785a074b Note the NPS arrowhead logo's official Pantone/RAL colors
CI / check (push) Successful in 38s
CI / deploy (push) Successful in 12s
Unrelated to anything the app renders, but the doc already collects
NPS-ecosystem color facts and this came up while sourcing colors
for a 3D print of the shield.
2026-09-21 18:08:35 +02:00
randy b0c93d10d7 Detect a LiveTrack session ending the moment Garmin confirms it, not 30min later
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 12s
Watched the session from the last two commits actually get saved:
Garmin updated its `end` field from the future 24h placeholder to
within a second of the last point's own time, immediately — much
faster than the recency-of-last-point fallback alone would have
noticed (that only flips after ACTIVE_WITHIN with no new points, so a
just-saved session with a last point seconds old would still read as
active for up to 30 more minutes).

Now checks end-already-in-the-past first (authoritative and near
instant when Garmin has set it), falling back to point recency only
when end still shows the future placeholder — the case an abandoned,
never-saved session never updates.
2026-09-20 16:55:57 +02:00
randy 3106f8c103 Draw a LiveTrack session's planned course alongside the actual track
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 13s
Garmin's own LiveTrack page already shows both when the activity was
started from a saved course — the session payload carries the course's
geoPoints right alongside the track points (LiveTrackClient found it
nested under the same React Query cache the session/points come from).

Rendered as a second, dashed LineString distinct from the actual
track's own line ("kind": "actual" vs "planned" on each GeoJSON
feature) — muted gray rather than matching Garmin's own purple-for-
planned, since this app's actual track is already purple and reusing
it would make the two indistinguishable.
2026-09-20 16:54:24 +02:00
randy 8eada07869 Drop subject:LiveTrack from the Apps Script's Gmail search, it finds nothing
CI / check (push) Successful in 21s
CI / deploy (push) Successful in 12s
Documented as matching regardless of language, on the assumption
Gmail's subject: operator does a substring match — empirically false
against a real, currently-arriving LiveTrack email whose subject does
contain the word ("Sledujte LiveTrack používateľa ...", confirmed
2026-09-20): with the filter, the search returned zero threads;
removing it and rerunning found the email immediately. Left out
entirely rather than replaced — from:noreply@garmin.com plus the
session-link regex already narrows this down to real LiveTrack emails
with something usable in them.
2026-09-20 16:44:20 +02:00
randy 5d86cb1d04 Detect a live LiveTrack session from its last point, not session["end"]
CI / check (push) Successful in 40s
CI / deploy (push) Successful in 13s
session["end"].nil? was the active signal since this was first built,
on the assumption Garmin leaves it unset until the activity actually
stops. Checked directly against a real still-recording session today
(2026-09-20): end was already non-nil, set to exactly start + 24h — the
share link's own expiry, not an end time — so every currently-active
session was showing as "Ended". A genuinely ended session's end does
hold its real end time (confirmed against the 2026-09-06 session this
was originally, and only, verified against), which is what made the
bug easy to miss.

Recency of the last track point is a directly-observable signal
instead of reverse-engineering what a given `end` value means: Garmin
posts one roughly every postTrackPointFrequency seconds (~15, observed)
while genuinely still recording.
2026-09-20 16:42:52 +02:00
randy b7618edc25 Move the session-skip from a before filter to real Rack middleware
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 12s
The before filter added in the previous commit never ran for static
assets: Sinatra serves public_folder files (every JS/CSS/font/vendor
file) through its own built-in shortcut, bypassing route and filter
dispatch entirely — confirmed live, curl still got a fresh set-cookie
on /js/map.js after that commit deployed, and the user hit the exact
same logout on refresh again. A HAR from that retest shows the same
clobbering pattern as before, just via a different response each time.

SkipSessionWrite is a plain Rack middleware instead, positioned (via
`use`, after `set :sessions`) inside Rack::Session::Cookie's own
wrapping — it sees the same env static dispatch does, so every response
is covered, not just Sinatra-routed ones.
2026-09-16 18:03:28 +02:00
randy bd46b72144 Skip re-writing the session cookie on every response, not just the three that change it
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 12s
Rack::Session::Cookie re-serializes and re-sets the cookie on every
response that even reads session — Rack::Protection's CSRF check does,
on nearly every request — reflecting whatever cookie that particular
request happened to arrive with, whether or not anything actually
changed. A HAR capture caught the real failure mode: seconds after a
successful login, offline_data.js's _checkUpdate (an unrelated
background poll fired the instant the page loads) was still carrying
the browser's pre-login cookie — a client-side timing race across the
OAuth redirect chain, not something this server controls — and its own
re-issued Set-Cookie landed after the login's, silently overwriting
session[:user] with nothing logged anywhere to explain it.

Only /login, /auth/callback and /logout ever actually write to
session. A before filter now skips the write everywhere else, removing
every other response from that race — a stale request can still read a
stale cookie, but it can no longer clobber a fresher one.

Also drops the earlier except: [:session_hijacking] from the previous
commit: rack-protection 4.2.1 only enables SessionHijacking when
explicitly opted into via :use, so Sinatra's default setup never turned
it on in the first place — the exclusion was a no-op, not the fix.
2026-09-16 17:56:44 +02:00
randy 6126b5055b Disable Rack::Protection::SessionHijacking, it silently logs everyone out
CI / check (push) Successful in 37s
CI / deploy (push) Successful in 19s
Sinatra turns this on by default once sessions are enabled: it pins a
session to the User-Agent header seen on whichever request first wrote
to it, and empties the session — no exception, nothing logged — on any
later request where that header doesn't match. That's exactly what
production hit behind its reverse proxy: OAuth login set session[:user]
fine, then the very next page load (even a plain refresh) came back
logged out with nothing in the logs to explain why.

Its own docs concede every header it checks is spoofable anyway ("will
not prevent determined hijacking attempts"), so losing it is a minor
tradeoff for a login that actually stays logged in. Every other
Rack::Protection check stays on.
2026-09-16 16:11:01 +02:00
randy 9edc2d9534 Show clicked hiking trail info in a docked panel, not a popup
CI / check (push) Successful in 22s
CI / deploy (push) Successful in 12s
A map-anchored Popup on a trail click routinely overlapped the very
line it described. Follows the same click -> docked panel pattern
gpx.js already uses for a track line click (TrackProfilePanel) rather
than inventing a new one: a fixed #trail-panel docks to the right edge
on desktop and the bottom edge on narrow viewports (this file's first
@media breakpoint), so the trail stays fully visible either way.

routeHtml()'s own markup is reused unchanged for the panel body —
.trail-panel__body's padding matches .maplibregl-popup-content's own
(15px 10px) so the header's canceling negative margin still lines up.
The popup-identity staleness guard for the async Overpass lookup is
replaced with a plain request token, since there's no longer a fresh
Popup instance to key off of.
2026-09-15 12:52:54 +02:00
randy eada464fb3 Highlight trails in their own color instead of recoloring to yellow
CI / check (push) Successful in 22s
CI / deploy (push) Successful in 12s
2026-09-13 20:17:48 +02:00
randy ca7772dd72 Highlight the clicked trail(s) on the map, both online and offline
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 12s
The real app had no visual feedback for which line a click actually
matched, unlike the earlier SVG prototype's hover highlight — just a
popup. Draws the matched route's real geometry (from the offline
vector feature, or reassembled from the online Overpass response's own
member-way geometry) as a separate yellow highlight layer, cleared
when the popup closes.

Guarded against a stale popup's closeOnClick "close" event clearing a
highlight that a newer click already set in the same event dispatch —
tracks which popup is currently "active" and only clears on its own
close, not a predecessor's delayed one.
2026-09-13 19:28:51 +02:00
randy 4c95927231 Pin check/renovate back to brick-build now that its image has Node
CI / check (push) Successful in 38s
CI / deploy (push) Successful in 12s
The runner's label-to-image mapping got repointed at Gitea's own
official runner image (docker.gitea.com/runner-images:ubuntu-latest,
has Node out of the box) — the earlier opensuse/leap:16.0 mapping for
brick-build specifically was what broke check/renovate on 2026-09-12,
not brick-build the label itself. Back to pinning both to this
machine specifically instead of whichever runner answers to the
shared ubuntu-latest label.
2026-09-13 12:32:14 +02:00
randy c6e4a455b6 Split offline hiking trails per country, add a picker instead of one big file
CI / check (push) Successful in 21s
CI / deploy (push) Successful in 11s
data/hiking_trails.geojson is now data/hiking_trails/<CODE>.geojson —
one file per country instead of a merge, generated per-source
(PmtilesBuilder.extract_hiking_trails), cached the same way
geofabrik_cache is (skipped once a country's own extract is already
newer than its source .osm.pbf, so growing the selection by one
country only re-extracts that one). GET /api/hiking-trails now lists
available country codes instead of a single file's status.

The Overlays panel's "Hiking trails" item becomes an expandable
per-country checklist once any offline extract exists (same shape as
Radar's own expand), replacing the plain on/off checkbox — nobody
covering a dozen countries' worth of offline map data needs all of
their hiking trails loaded and parsed at once, so each ticked country
fetches its own GeoJSON independently instead.

HikingTrails.extract_merged (server-side multi-country merge, no
longer needed now that the client picks per country) removed again.
Verified the new extract_hiking_trails + caching skip live on brick
against a real cached country PBF.
2026-09-13 11:17:05 +02:00
randy 079ac37787 Wire offline hiking trails into the real build, API, and renderer
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 12s
data/hiking_trails.geojson now generates automatically as the last
step of every offline.pmtiles build (PmtilesBuilder.extract_hiking_trails,
best-effort — a failure here doesn't fail the whole build), merged
across all selected countries with cross-border dedup by OSM relation
id (HikingTrails.extract_merged). Served at GET /tiles/hiking_trails.geojson
and GET /api/hiking-trails, fetched client-side only once the overlay
is actually toggled on.

hiking_trails.js now prefers this local file whenever it exists,
regardless of which base map (online/offline) is active — same
"prefer local when available" story as HillshadeControl already uses
for terrain. Offline click hit-tests the real vector layer directly
(queryRenderedFeatures) instead of querying Overpass — instant, no
network. Coordinates are rounded to 5dp to keep the combined file
smaller; colour tag values are normalized against a known-safe set
before ever reaching a style paint property.

Verified extract_merged (new: combines multiple countries' extracts,
dedupes overlapping relations) against two real cached country PBFs
on brick directly, not just fixtures.
2026-09-13 10:31:59 +02:00
randy b35ace9734 Fix hiking trails extraction: add-locations-to-ways was missing
Verified live on brick against real cached extracts: a plain Geofabrik
PBF's ways only carry bare node references, not coordinates, so
locations_on_ways on the final `cat -f opl` had nothing to embed —
every way came out as e.g. n123xy, empty x/y, and every relation got
dropped for having no usable geometry (0 routes for Macedonia).

osmium add-locations-to-ways between the tags-filter and cat steps
resolves those references first. Re-verified after the fix: 45 routes
for Macedonia, 3009 for Slovakia, with real colour/ref/operator/
distance (326 red, 782 yellow, 510 blue, 902 green KST-style routes).
2026-09-13 09:26:22 +02:00
randy 714795d52b Prototype: extract offline hiking trails from the Geofabrik PBF we already have
CI / check (push) Successful in 43s
CI / deploy (push) Successful in 13s
No custom Planetiler profile needed after all — Maptool::HikingTrails
runs osmium tags-filter r/route=hiking against the same .osm.pbf
PmtilesBuilder already downloads per country for offline.pmtiles, then
osmium cat -f opl (locations_on_ways=true) to get member-way geometry
back as plain text, no separate node-id lookup table to build.

No general OSM tool stitches a route relation into one line (routes
are a semantic grouping, not a geometry type) — parse_opl does that
walk itself, same as the Overpass-based click handler already does
client-side. Kept pure/testable apart from the two osmium subprocess
calls, mirroring OverpassPois.fetch vs .to_geojson.

rake hiking:trails[pbf,out] to try it by hand against a real cached
extract. Not wired into the actual offline build or rendering yet —
still just a prototype, per TODO.adoc.
2026-09-13 09:18:50 +02:00
randy cc3d8ca5f3 Race 3 Overpass mirrors for speed, show more per-route detail
CI / check (push) Successful in 2m18s
CI / deploy (push) Successful in 11s
The single shared overpass-api.de instance varies wildly in load —
Promise.any against three independent public mirrors (also CORS-open,
verified) means only the slowest one determines wait time, not
whichever happens to be hit.

Also show the description tag (falls back to it for the title when a
route has no name, e.g. many KST route relations), the ref number, and
a "View on OpenStreetMap" link — the popup was skipping data that was
already in the response for free.
2026-09-13 08:53:48 +02:00
randy 6f1577f957 Click a hiking trail to see its name, color, network and distance
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 12s
The Waymarked Trails raster overlay has no queryable features (just
pixels), so this answers "what trail is this" with a one-off Overpass
query on click instead: way(around:40m) finds candidate ways, then
rel(bw) walks backward to their parent route=hiking relations. Overpass
has permissive CORS, so this runs straight from the browser, no server
proxy needed — verified against a real KST route in the Tatras.

Route length isn't tagged directly, so it's summed from each member
way's own geometry (measure.js's existing haversine lineLength, now
exported and reused rather than duplicated). The colour tag is only
ever used as a CSS background after matching a strict hex/word regex —
untrusted free text otherwise.
2026-09-12 23:32:56 +02:00
randy 9c45e7275c Add a Hiking trails overlay (Waymarked Trails, online only)
CI / check (push) Successful in 2m16s
CI / deploy (push) Successful in 11s
New Overlays panel toggle, same plain on/off shape as Contour lines —
raster tiles from waymarkedtrails.org, which already color each route
by its OSM colour tag (the KST/KČT paint-blaze colors used in
Slovakia/Czechia, plain red elsewhere), so no styling work needed here.

Online only for now; offline (baked into offline.pmtiles) needs a
custom Planetiler profile for route relations, deferred until there's
a route planner/router to pair it with.
2026-09-12 22:36:10 +02:00
randy c894706140 Revert check/renovate to ubuntu-latest — brick-build's image has no Node
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 12s
brick-build's job container (opensuse/leap:16.0) has no Node.js, and
every step in check/renovate (checkout, setup-ruby, setup-node) is a
JS-based Action that needs it just to start — both runs since the
brick-build switch failed immediately, and deploy never ran because it
depends on check passing. ubuntu-latest is shared with "Gitea Runner on
Cube" (not pinned to this specific machine), but that's fine: only
deploy (a unique label held by just brick's native runner) actually
needs to land on a specific box.
2026-09-12 21:39:21 +02:00
randy 21988c61da Lower offline terrain to z13, not z15 — raster DEM storage explodes with zoom
CI / check (push) Failing after 4s
CI / deploy (push) Skipped
The 12-country selection's terrain.pmtiles was headed well past the
181GB an earlier 5-country build already produced at z15. Raster DEM
tiles don't compress away like vector ones (every tile is a real
image, even over empty terrain), so each extra zoom level roughly
quadruples storage. z13 is each Mapterhorn regional chunk's own native
minzoom — one raster level per chunk instead of a 15-13=2-level
pyramid on top — still enough for the hillshade toggle and the planned
slope readout. Online hillshade keeps its own z15 cap; this only
affects the offline copy.
2026-09-12 20:18:36 +02:00
randy f18e5c19d3 Truncate noisy build-failure output, fix the recurring terrain death
CI / check (push) Failing after 4s
CI / deploy (push) Skipped
The web UI was showing a raw ~500k-character stdout/stderr dump on a
failed build — go-pmtiles' extract logs one progress line per chunk,
which is fine on a terminal (it overwrites itself) but not captured
non-interactively. command_output now keeps just the tail, and the
status line renders it with preserved line breaks in a scrollable box
instead of one unreadable blob.

Root cause of the actual failure (terrain build dying ~11 minutes in
with an HTTP/2 stream reset): go-pmtiles' default --overfetch=0.05
merged this multi-country planet.pmtiles extract into a single ~60GB
request with no way to resume after a dropped stream. --overfetch=0
splits it into many smaller requests instead.
2026-09-12 19:41:45 +02:00
randy 237e5b2e54 Move check/renovate onto brick-build, deploy stays brick-host
Renovate no longer needs Docker access for a one-shot docker-run job:
switched to setup-node + npm install -g renovate (matching labelprint's
workflow), so it fits the same brick-build runner as check instead of
the stale "deploy" label. Also note down the build-log readability/API
gaps flagged from a real failed terrain build.
2026-09-12 19:36:08 +02:00
randy 3f9c46e846 Skip rebuilding offline/terrain data when nothing actually changed
CI / check (push) Successful in 21s
CI / deploy (push) Successful in 12s
2026-09-12 13:32:02 +02:00
randy 17d1adedbc Guard syncTerrain against running before the style has loaded
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 12s
2026-09-12 13:27:53 +02:00
randy cd02b03502 Retry pmtiles extract/merge on failure, not just on the first try
CI / check (push) Successful in 39s
CI / deploy (push) Successful in 12s
2026-09-12 12:50:09 +02:00
randy 5c3151c2f5 Add a 3D pitch toggle, with raised terrain following it 2026-09-12 12:50:09 +02:00
randy f5eb9a4a8d Tell Planetiler explicitly how many CPUs it has
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 12s
Its own autodetection was leaving most of a 16-core host idle
(observed in production) — likely a container cgroup/JVM detection
quirk. Etc.nprocessors asks the OS directly instead.
2026-09-12 01:34:29 +02:00
randy f09b5de6c2 Relabel data bind mounts for SELinux (:Z)
CI / check (push) Successful in 21s
CI / deploy (push) Successful in 12s
App crashed on startup on brick (Permission denied @ dir_s_mkdir -
/app/data/gpx) — brick has SELinux enforcing, unlike cube, and Podman
denies a container write access to a bind-mounted host directory
without a relabel regardless of Unix permissions. Applied to both
maptool's ./data mount and kiwix's ZIM mount (same underlying cause,
would have hit this too once a real ZIM is placed there).
2026-09-12 01:27:23 +02:00
randy 0080ac6170 Deploy to brick-host instead of cube-host
CI / check (push) Successful in 19s
CI / deploy (push) Failing after 5m1s
cube.local and brick.local swapped IPs and brick is now the deploy
target — rename the native runner label everywhere it's referenced.
2026-09-12 01:15:09 +02:00
randy 78362d51f2 Show what's actually cached per-country, with a delete button per row
CI / check (push) Successful in 17s
CI / deploy (push) Successful in 13s
The only visibility into downloaded data was the whole offline.pmtiles
row, and the only delete was "everything" (whole file + whole cache).
GET /api/offline-tiles/cache lists what's really in
data/geofabrik_cache/ right now — distinct from offline.pmtiles' own
meta.sources, which only reflects the last successful build and can
disagree with both the current selection and the cache (ticked-but-
not-Applied, or built-from-but-since-unticked).

The region picker now shows each cached country's size next to its
checkbox, with its own delete button (DELETE
/api/offline-tiles/cache/:code, already added but unused by the UI
until now) — removing just that entry rather than clearing the whole
cache.
2026-09-11 23:39:43 +02:00
randy d09d953cbf Pass --download to Planetiler for its auxiliary OpenMapTiles sources
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 13s
Real build failure once the tmp-filename fix got past the format-
detection error: "data/sources/lake_centerline.shp.zip does not exist.
Run with --download to fetch it". The OpenMapTiles profile needs a few
small non-OSM sources (water polygons, Natural Earth, lake
centerlines) alongside the .osm.pbf itself, and Planetiler only
fetches them when told to. They land under its own default
data/sources/, gitignored alongside the rest of data/ — fetched once,
reused by every later build.
2026-09-11 23:28:07 +02:00
randy b54020e23b Fix Planetiler tmp-file format detection, add build progress and delete
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 13s
Root cause of "planetiler exited with status 1: ... Unsupported format
tmp" (and likely the earlier unexplained "pmtiles exited with status
1"): Planetiler/go-pmtiles pick their output archive format from the
file extension, and the atomic-rename tmp files were named
"offline.pmtiles.tmp"/"terrain.pmtiles.tmp" — ".tmp" as the extension,
not ".pmtiles". Renamed to "offline.tmp.pmtiles"/"terrain.tmp.pmtiles".

Also, since build-status only ever said "running since X" with no way
to tell what it was actually doing:
- GET .../build-status now includes a `step` string ("fetching PL
  (3/4) — done: SK, CZ — pending: HU", "tiling with planetiler", ...)
  while a build is in progress.
- DELETE /api/offline-tiles / /api/terrain-tiles clear a built file,
  its sidecar meta, and (for offline) the whole per-country download
  cache — a manual "reclaim space" / "start over" action the region
  picker's Apply never did on its own. DELETE
  /api/offline-tiles/cache/:code clears just one country's cached
  extract. All three are 409 while that kind's build is running, and
  surfaced as a small "Delete" button on each row in the Offline data
  panel.
2026-09-11 23:23:46 +02:00
randy 2878270187 Include stdout in run_*! build error messages, not just stderr
Real cause of "pmtiles exited with status 1:" with nothing after the
colon: go-pmtiles (like some other CLIs) can write its actual error to
stdout, which was being silently discarded. Applies the same fix to
osmium/planetiler for consistency.
2026-09-11 23:07:49 +02:00
randy 007dee7341 Cache per-country Geofabrik extracts so Apply doesn't redownload everything
Every region-selection change re-downloaded every selected country's
.osm.pbf from scratch, even ones already fetched by a prior build —
painful for multi-GB countries. Extracts now persist in
data/geofabrik_cache/, keyed by country, and are only re-fetched when
Geofabrik's Last-Modified for that country actually moved on (checked
with a cheap HEAD) or the file isn't cached yet.
2026-09-11 22:52:16 +02:00
randy 077b44ece1 Note OpenTopoMap-style forest/meadow fill patterns as a TODO idea
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 2m54s
2026-09-11 22:13:30 +02:00
randy 619f2293df Move Docker base image to Debian trixie for Java 21
Planetiler 0.10.2 needs Java 21+; bookworm's default-jre-headless (and
its backports) only goes up to 17, causing a real build to fail with
UnsupportedClassVersionError. trixie's default-jre-headless is Java 21.
2026-09-11 22:13:30 +02:00
randy a84e6ae22f Follow Geofabrik's 302 redirects on extract download and Last-Modified check
CI / check (push) Successful in 29s
CI / deploy (push) Successful in 16s
Geofabrik redirects europe/<slug>-latest.osm.pbf to a dated filename
(confirmed live: .../europe/austria-latest.osm.pbf -> .../europe/austria-
260910.osm.pbf) — a real build hit this immediately. Net::HTTP doesn't
follow redirects on its own; both the streaming download and the HEAD-based
update-check now do, up to a small limit.
2026-09-11 20:57:22 +02:00
randy 920a1d2d36 Stream Geofabrik extract downloads instead of buffering in memory
Net::HTTP's plain #get loads the whole response body before returning —
fine for the small JSON manifests elsewhere in this file, but a real risk
for a multi-GB country .osm.pbf (Germany, Russia, ...). Streams to disk
via read_body instead.
2026-09-11 20:56:06 +02:00
randy d08ed41185 Replace offline basemap pipeline with Planetiler, unify online/offline styling
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 2m43s
Offline tiles were built from Protomaps' own basemap schema, needing a
completely separate hand-tuned theme (protomaps-themes-base + OFFLINE_THEME)
from the online Liberty style's ONLINE_STYLE_PATCH — the two could never be
made to match exactly no matter how much either side got tuned (today's
session found and fixed several mismatches: building opacity, park
boundaries, road casings, road widths).

Offline.pmtiles is now built with Planetiler's default OpenMapTiles profile
against Geofabrik per-country OSM extracts (osmium-merged when more than one
country is selected), the same vector schema Liberty already expects.
offlineStyle() now clones the same Liberty JSON (vendored as a local
snapshot for reliability with no internet) and applies the identical
ONLINE_STYLE_PATCH, just repointed at local PMTiles/glyphs/sprite — one
palette instead of two, correct on both sides by construction.

Also vendors Liberty's icon sprite for the first time (offline had none
before), adds Java/osmium-tool/planetiler.jar to the Docker image, and
removes the now-unused protomaps-themes-base dependency and the
theme-comparison debug page that found today's bugs.
2026-09-11 20:52:11 +02:00
randy 0874e8f8ed Add a debug page for comparing offline base-map themes side by side
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 13s
Renders the bare protomaps-themes-base themes next to the app's actual
offline/online styles (each real app instance, embedded), with pan/zoom
kept in sync via the existing URL-hash mechanism, to make it easy to see
where the hand-tuned offline palette diverges from the online one.
2026-09-11 16:58:32 +02:00
randy 9223410b40 Swap flag emoji for real (square, bordered) SVG flags
CI / check (push) Successful in 21s
CI / deploy (push) Successful in 16s
Emoji flags are font glyphs, not shapes — no way to resize, border, or
tone them down to fit this app's controlled, hand-drawn-icon look
(same reasoning the topbar readout already settled earlier this
session). Vendored flag-icons' pre-cropped square SVGs (MIT) instead,
framed in a small rounded-square badge — the same convention
icons.js's MAP_MARKER_ICON already uses for framing a glyph, not a new
visual language.
2026-09-11 13:31:36 +02:00
randy 4a3862189f Track what was actually built, and flag when it's stale
CI / check (push) Successful in 19s
CI / deploy (push) Successful in 16s
The .pmtiles files themselves carry no record of what upstream
snapshot they came from — only a local mtime. Each successful build
now writes a small sidecar (offline.pmtiles.meta.json: the Protomaps
build date used; terrain.pmtiles.meta.json: the Mapterhorn source
names/md5sums) recording that, surfaced in the panel next to the
existing size/updated line.

New GET .../check-update routes compare the recorded build against
the live upstream index/manifest — true/false/null, null meaning
"don't know" (no local build yet, or the check itself failed) rather
than a false "no". Kept separate from the plain status endpoints
(which map.js's own topbar dot hits on every page load) since this
one reaches an external service; only called once when the Offline
data panel is actually opened.

Also: flag emoji next to each country in the picker (derived from the
ISO code via Unicode regional indicators, no image assets needed).
2026-09-11 13:18:32 +02:00
randy 53c9a82af2 Add a country-based region picker for offline data
CI / check (push) Successful in 17s
CI / deploy (push) Successful in 12s
Replaces "regenerate for the current viewport" with what Locus/OsmAnd-
style apps give you: tick which countries you want offline (a Natural
Earth-derived checklist, public domain, vendored), Apply rebuilds
offline.pmtiles/terrain.pmtiles to cover the whole selection at once.
Multiple regions can be offline simultaneously (e.g. Slovakia + Austria
for a trip that crosses the border) — the selection itself is the
source of truth (data/offline_regions.json), not a set of per-country
files, so "remove a country" and "refresh a country" are both just
"change the selection, then Apply."

Confirmed against go-pmtiles' own source (not just its docs) before
committing to this shape: `extract --region=<geojson>` treats a
FeatureCollection's features as their union already (no dissolve/union
math needed on our side — pmtiles/region.go's UnmarshalRegion just
concatenates every feature's rings), and `merge` hard-errors on any
overlapping tile id between inputs (merge.go's prepareInputs). That
second fact rules out "extract each country separately, merge them
all" — adjacent countries' region-extracts would share border tiles,
since extraction copies whole tiles rather than clipping vector
content to the polygon. The actual fix: extract per *source archive*
(the Protomaps build, each intersecting Mapterhorn chunk), each call
covering the whole current selection via one --region= file — the
terrain pipeline already extracted-then-merged per source this way, it
just needed a multi-country region instead of a single bbox.
2026-09-11 12:55:56 +02:00
randy f89dea5805 Build terrain.pmtiles from Mapterhorn instead of fetching tiles ourselves
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 1m52s
Mapterhorn (mapterhorn.com, Protomaps-affiliated) publishes pre-built
Terrarium-encoded terrain as PMTiles: a global planet.pmtiles (z0-12)
plus ~450 regional chunks (z13+). Same pmtiles extract --bbox=...
mechanism offline.pmtiles already uses, just against their files —
confirmed live against their real manifest and the documented
Interlaken example. Replaces fetching every individual tile ourselves
(even after this session's earlier concurrency fix, still tens of
minutes for a real region) with 1-3 extract calls plus a merge when
the bbox spans more than one chunk: seconds, not minutes.

Removes CONCURRENT_TILE_FETCHES, write_terrarium_mbtiles, tile_range,
lng/lat_to_tile_x/y, fetch_terrarium_tile, and the sqlite3 gem
entirely — none of it is needed once the tiles arrive pre-packaged.

dem.js's point-elevation lookup (shadow.js's terrain-shadow overlay)
hardcoded a 256px PNG tile assumption that Mapterhorn's 512px WebP
tiles would have silently broken — now reads the actually-decoded
tile's own size and mime type instead of assuming either.
2026-09-11 12:26:03 +02:00
randy d041c21270 Fetch terrain tiles concurrently with persistent connections
CI / check (push) Successful in 20s
CI / deploy (push) Successful in 13s
Serial fetches (no connection reuse) measured ~650ms/tile, almost all
TCP+TLS handshake — a real bbox at z0..15 is tens of thousands of
tiles, so this was hours for a modest region. Now: 16 worker threads
share a queue (I/O-bound, so the GVL doesn't get in the way), each
keeping its own persistent connection across every tile it fetches.
Measured live against the real Terrarium source: ~40ms/tile, ~16x
faster.

write_terrarium_mbtiles's SQLite writes and progress updates stay on
one thread (draining a results Queue the workers feed), so nothing
needed its own locking beyond the two Queues themselves.
2026-09-11 12:04:13 +02:00
randy bbe9f38bda Show build progress instead of a static "Building…"
CI / check (push) Successful in 18s
CI / deploy (push) Successful in 13s
terrain.pmtiles can take minutes (tiles fetched one at a time) with no
way to tell a slow build from a stuck one. Now reports tiles_done/
tiles_total into the same status Hash the frontend already polls, and
the panel shows elapsed time (both files) plus tile counts (terrain
only) so the numbers visibly move every ~3s poll instead of sitting on
one static string.
2026-09-11 11:57:43 +02:00