A logged-in user can now trigger, from a new "Offline data" sidebar
panel, what was previously a fully manual go-pmtiles workflow
(docs/development.adoc): regenerate either file for the current map
viewport, no area-picker (same "visible area is the area" convention
the Overpass POI plugin already uses).
- lib/maptool/pmtiles_builder.rb: two independent background jobs
(Thread + Mutex-guarded status Hash, same shape as app.rb's own GNSS
poller), each writing to a .tmp path and atomically renaming into
place so a build never serves a partially-written file mid-run.
offline.pmtiles is a single `pmtiles extract` against Protomaps'
public daily basemap build (byte-range extraction, not a full
~120GB download). terrain.pmtiles has no equivalent public
Terrarium-encoded archive to extract from, so it fetches individual
Terrarium tiles itself, packs them into an intermediate MBTiles
(new sqlite3 gem), then the same `pmtiles convert`.
- 4 new app.rb routes (POST .../build, GET .../build-status) plus
size/mtime added to the existing availability endpoints.
- public/js/offline_data.js: new sidebar panel, polling build-status
the same self-rescheduling-setTimeout shape livetrack.js already
uses for its own refresh loop.
- Both network-heavy pieces (the Protomaps build index, per-tile
fetches) are injectable, so test/test_pmtiles_builder.rb exercises
the actual build pipeline against a fake `pmtiles` binary without
hitting a real network, matching this suite's existing
keep-it-offline convention for external services.
The topbar's coordinate display was plain decimal degrees ("48.14611,
17.17722") — switched to DMS (48°08'46"N 17°10'38"E), the more
GPS-unit-like format Locus Pro itself uses, and made it click-to-copy
(plain decimal, since that's what pastes cleanly into another map/GPS
app — not the DMS text actually shown).
Added sunrise/sunset + a countdown to whichever's next, for the same
map-center point the coordinates already describe. Vendored SunCalc
(BSD-2-Clause, github.com/mourner/suncalc) rather than hand-rolling
the astronomical math — small, dependency-free, and correctness here
has real pitfalls (equation of time, refraction, polar day/night) a
first-pass reimplementation would risk getting subtly wrong.
Also added a test asserting every public/js/*.js module has a
versioned importmap entry in layout.erb — caught that overlays.js
(from the previous commit) was missing one, which would have quietly
skipped ASSET_VERSION's cache-busting for that file after every
deploy.
The old link only ever surfaced one nearby article at a time, buried
in whatever POI's popup you happened to open, and only for uploaded
files with a category. Replaced it with a standalone "Wikipedia"
toggle under the renamed "Tracks & Points" panel: enabling it adds a
live map layer that refetches GET /api/kiwix/nearby on every pan/zoom
and plots every nearby article as its own point.
kiwix-serve's own geo search never echoes back a result's coordinates
(confirmed live against the real production ZIM, both its HTML and
?format=xml output) — it only filters by distance from a point. So
Maptool::Kiwix.nearby fetches each matching article's own page too and
reads its <meta name="geo.position"> tag, which every geotagged
Wikipedia article carries. Tried parallelizing those per-article
fetches with threads first; measured no improvement against the real
ZIM (kiwix-serve appears to serialize requests regardless), so it
stayed sequential — ~1.6s for a 30-result page, fine for a
refetch-on-pan-end layer.
Clicking a point opens a small popup linking to the article (proxied
through the existing /kiwix/* passthrough). New WIKI_ICON (NPS's
bookstore-22.svg) marks these, kept out of POI_ICONS since that set
doubles as notes.js's icon-picker option list and this fixed-glyph
layer has no per-feature category to key off of.
- notes.js's ICON_LABELS wasn't extended for the shop_supermarket/
shop_convenience keys icons.js added — the note icon picker showed
the raw internal key as the title instead of a real label.
- Plugins.run mislabeled a malformed plugin.json as "plugin did not
print valid JSON" — a message written for bad script stdout, not a
bad manifest, since both paths shared one rescue.
- overpass_pois.rb's description_for joined raw OSM tag values with
"\n" — an operator/opening_hours tag with an embedded newline could
inject an extra line into the popup that gpx.js's formatDescription
(built for p4n2gpx's own well-formed lines) would render as its own
independent, legitimate-looking link. Tag values are stripped of
embedded newlines before joining.
plugins/ ships baked into the image and gets wiped/replaced by the
checkout sync on every deploy (docs/deployment.adoc), so a private
plugin dropped there — the exact use case the plugin contract was
designed for — didn't survive a redeploy the way uploaded GPX/kiwix
data under data/ does.
Plugins.list/run now accept either one directory or an array, and
app.rb checks the built-in plugins/ plus data/plugins/ (already
covered by the existing ./data bind mount, no new volume needed).
Overpass categories now carry the same icon+color styling a p4n2gpx
waypoint gets, instead of falling back to gpx.js's plain red marker,
plus a popup description built from opening_hours/operator/website
tags. Supermarket/convenience store reuse NPS's generic store glyph,
the only honest match in that icon set.
Each plugin run's saved filename now bakes in the area's center and a
timestamp, so repeated downloads for different areas land as separate,
identifiable files instead of overpass-pois.geojson/-1/-2/...
A plugin is a directory under plugins/ with a plugin.json manifest
({"label": "...", "command": "./run.sh"}) plus a script that reads
--south/--west/--north/--east and prints a GeoJSON FeatureCollection
to stdout (lib/maptool/plugins.rb) — that's the entire interface a
plugin needs to know about this app. The sidebar's new import icon
(logged in only, GET /api/plugins + POST /api/plugins/:name/run) lists
whatever's found and runs one for the current map viewport, saving its
output as a new .geojson file the same way an upload would be.
The point: a scraper for a site whose ToS forbids that (Park4Night,
iOverlander, Freecampsites) doesn't need to live in this public repo
to work here — its own directory just goes under plugins/ locally and
is never git add'ed, no special-casing needed anywhere.
plugins/overpass-pois wraps the existing rake overpass:pois query as
the first, in-repo, reference example of the contract.
Also fills in four modules (auth.js, locate.js, measure.js, notes.js)
missing from layout.erb's cache-busting importmap alongside adding
plugins.js's own entry — a pre-existing gap that meant they could keep
serving a stale cached copy across a deploy despite the importmap
existing specifically to prevent that.
Sinatra decodes a route's splat before handing it to the block, so an
article link like /kiwix/content/wikipedia/Dlhé_diely arrived as raw
UTF-8, not percent-encoded. URI#path= rejects that outright
(URI::InvalidComponentError) rather than escaping it, which .proxy's
rescue turned into a silent nil -> every article link 502'd, while the
search results page linking to them worked fine (its own path,
/kiwix/search, is plain ASCII).
map.priesol.net sits behind a reverse proxy that only forwards to
maptool itself, so the previous approach (publish kiwix-serve's port,
link the browser straight at it) had no address a real browser could
ever reach — not just a Docker-internal hostname as first assumed, but
no host/port combination at all works from outside.
GET /kiwix/* now proxies to kiwix-serve (Kiwix.proxy in
lib/maptool/kiwix.rb, a plain Net::HTTP passthrough), and the kiwix
service runs with --urlRootLocation=/kiwix so every link/asset it
emits already carries that same prefix and needs no rewriting
(confirmed against libkiwix/internalServer.cpp, which strips this
exact prefix off incoming requests). Drops the port-publishing
approach from the previous commit entirely.
lib/maptool/kiwix.rb: a small reachability check for an external
kiwix-serve instance. GET /api/kiwix reports it (same optional-
integration pattern as offline-tiles/terrain-tiles); the frontend
adds a "Nearby on Wikipedia" link to a point's popup pointing straight
at kiwix-serve's own geo search (GET /search?latitude=&longitude=
&distance=, confirmed from libkiwix's actual source rather than
guessed) — linked to directly, not proxied/embedded, since it already
has a full search UI of its own.
docker-compose.yaml: new `kiwix` service (ghcr.io/kiwix/kiwix-serve)
on the default network, KIWIX_URL wired to reach it by service name.
The ZIM itself isn't part of the repo (large, self-contained archive)
— data/kiwix/ gitignored same as the *.pmtiles files, setup steps in
docs/development.adoc.
No kiwix-serve/ZIM available here to test end-to-end — verified the
app side against a stand-in local HTTP server (GET /api/kiwix flips
available correctly) and confirmed the generated popup link matches
the real geo-search query shape. docker-compose.yaml validated with
`docker-compose config`. Full test suite: 118 runs, 0 failures.
Measure already draws distance lines and area polygons by hand — a
second click-to-draw UI in the notes tool (Point/Area mode, from the
previous commit) duplicated that. Replaced it with a "Save as note"
button on the Measure panel that saves whatever shape is currently
drawn there; NotesControl goes back to being just a one-click point
pin. Both share a single NoteSaveForm (notes.js) for the note-text +
target-layer-file UI instead of two copies of it.
gpx.js: the "lines" source now also renders a note-flagged LineString
(dashed, popup-on-click) alongside the Polygon fill/outline from
before — the plain track-line layer excludes category:"Note" features
so a saved distance note doesn't also get the track hover-summary/
profile-panel treatment.
Notes are now individually editable/deletable from their own popup
(logged-in only): Edit swaps the popup content for an inline
textarea, Delete confirms and removes. Both target the feature by a
new client-generated properties.id rather than array position (new
GpxStore#update_feature/#delete_feature, PUT/DELETE /api/gpx/notes),
since neither array index nor the note's own text (the thing being
edited) is a stable identifier.
Verified live: Measure's Save button (disabled/enabled state, saves
the exact on-screen shape), the popup's Edit/Delete buttons and inline
edit form. Full test suite: 115 runs, 0 failures (new coverage for
update_feature/delete_feature and the PUT/DELETE routes).
notes.js: Point/Area mode toggle mirroring measure.js's UX — Area
mode accumulates clicked vertices with a live dashed-preview polygon
(own small preview source/layers), a Finish button once 3+ corners
are placed, then the same save form as Point mode.
gpx.js's _addLayers now also renders Polygon features from a file's
"lines" source (fill + outline, filtered by geometry-type — the
existing line layer already only draws LineString, so this is just
two more layers on the same source, not a new bucket) with
popup-on-click like a waypoint, not the track line's hover-summary/
profile-panel behavior. Without this a saved polygon wouldn't have
rendered back on reload.
Fixed along the way: .notes-panel__form/.notes-panel__modes both set
their own `display`, which silently wins over the browser's default
`[hidden] { display: none }` (author styles always beat user-agent
ones) — same gotcha .gpx-panel__action[hidden] already has a comment
about. The form was never actually invisible before being opened.
Verified live: real polygon file (fill/outline render, click ->
popup) and the full Area-mode draw flow (mode switch, live preview,
Finish, form) via headless Chrome. Full test suite: 104 runs, 0
failures — one new backend test confirms a Polygon feature round-
trips through POST /api/gpx/notes.
Known gap (noted in TODO.adoc): the category filter matrix only
re-slices the points source, so toggling "Note" off hides point notes
but not polygon ones.
lib/maptool/overpass_pois.rb: queries Overpass for drinking water,
dump stations, LPG refill, toilets, laundromats, supermarkets/
convenience stores, and springs within a bbox, writes GeoJSON. A
dev-time tool (like the offline.pmtiles/terrain.pmtiles generation
steps already documented) — not called by the running app, drop the
output into data/gpx/ or upload it through the Tracks panel like any
other .geojson layer.
Verified against the real Overpass API (88 POIs for central
Bratislava) in addition to the new unit tests (pure to_geojson/
build_query logic, no network in the suite itself). Full suite: 103
runs, 0 failures.
New map-corner "Add note" control (public/js/notes.js), mirroring
measure.js's click-to-place UX. Click the map, write a short note,
pick an existing .geojson layer file or name a new one — saved via a
new GpxStore#append_feature (grows the target file in place, unlike
save()'s always-disambiguate-the-filename behavior) through a new
POST /api/gpx/notes route.
No new rendering path: a note is just another point in a .geojson
file the Tracks panel's tree already lists, checkboxes, and pops up —
visibility toggling ("layers") comes for free from that existing
machinery, per-file, with no changes needed there.
Points only for now — a polygon mode would need gpx.js's point/line
split taught a third Polygon bucket first, or a saved polygon
wouldn't render back on reload (see TODO.adoc).
Covered by new tests: GpxStore#append_feature (create/append/missing-
path/filename-safety) and the /api/gpx/notes route (auth gate, create,
append, malformed/missing-geometry rejection, 404 on a missing
target). Full suite: 97 runs, 0 failures.
/api/health now reports REVISION alongside status, and the CI wait
step checks for this deploy's own short SHA specifically - a stray
old container that never got torn down (or a build that silently
kept serving stale code) would still 200 on a plain health check but
now won't match.
Ended sessions past the ~24h Garmin appears to keep a live-track page
around for get verified once at app startup and, once confirmed dead,
flagged so the panel greys them out instead of only failing after a
click. Already-flagged sessions are skipped on later startups so this
doesn't keep re-hitting Garmin for the same dead session.
GET /api/gnss/position now requires login (app.rb) -- it reveals
where the receiver physically is, not something to leave open to
anyone with the URL. GnssControl's sidebar icon is hidden entirely
when logged out (same pattern as the tree UI's write controls) rather
than shown and failing to ever get a fix.
LocateControl needs no changes: it only ever reaches GNSS through
GnssControl#isEnabled(), which can't turn true for a logged-out
visitor if the toggle is never on the page to click. Its
browser-geolocation fallback stays open to everyone -- a visitor's
own device asking for their own location isn't something to gate.
Extracted isLoggedIn() out of gpx.js into a shared public/js/auth.js
(gnss.js needed the same check) rather than duplicating it.
Verified: /api/gnss/position returns 401 logged out; the sidebar icon
is actually hidden (computed display: none, not just present-but-
unstyled -- checked properly this time after almost repeating the
same present-vs-visible test mistake the .gpx-panel__action bug
already taught).
?download=... on GET /api/gpx/file now requires login, same as
upload/move/delete/create-folder -- plain viewing (no ?download)
stays public. Added ?download=original alongside the existing
GeoJSON export, serving the untouched, as-uploaded file directly via
send_file -- the file name link still downloads GeoJSON, a new
button next to delete downloads the original.
Since both are real <a href> links (so right-click/middle-click still
work), a logged-out click can't just be left as a plain navigation --
it would land on a raw JSON error page instead of being sent to log
in. _download (gpx.js) fetches instead, reuses the existing
redirectIfUnauthorized 401 check, and only then turns the response
into a blob and clicks a throwaway <a download> to trigger the save.
Verified in a real browser: an anonymous click on either download
correctly redirects through /login to the real OAuth login page
instead of showing raw JSON.
Once a .gpx/.fit is cached, the tree shows it under its converted name
("ride.gpx" -> "ride.gpx.geojson", matching the real cache file next
to it on disk) instead of still reading as un-converted -- the
GeoJSON is what's actually on the map from that point on, and the
original is never deleted, just not re-shown as itself. The name is
now a real link (GET /api/gpx/file/<path>?download=1) instead of
plain text, the only way to get that converted form out of the app.
Verified in a real browser that an <a> nested in a <label> claims its
own click rather than the label forwarding it to the checkbox, so
downloading and toggling visibility don't fight over the same click.
A .geojson file needs no GPX/FIT conversion — it's already the shape
gpx.js expects — so GpxStore#geojson now just reads and returns it
as-is, no cache sidecar. save/tree/the upload dropzone and its zip
handling all pick it up through the existing TRACK_EXTENSIONS list.
Also fixes a bug the new "cached" tree icon would have shipped with:
a GPX/FIT's own .geojson cache sidecar (e.g. "ride.gpx.geojson") also
matches the new .geojson branch in list_children, which would have
listed it as a second, phantom tree entry — excluded explicitly, with
a regression test.
GpxStore#tree now reports a "cached" flag per file (reusing the same
mtime freshness check geojson() already does); the file tree shows a
small curly-braces icon next to any file whose conversion is already
on disk, so it's visible which files will toggle on instantly versus
which still need to parse.
A newer p4n2gpx run (Ruby rewrite, ~/test/mapping/p4n2gpx) started
emitting one plain GPX <link href> per park4night listing photo,
which this app parsed but never displayed. Gpx.waypoint_photos
collects them into a photos property; the waypoint popup renders them
as a row of thumbnails above the description, each opening full-size
in a new tab. Absent on older exports and plain GPS waypoints, so the
row just doesn't render for those.
GpxStore#save now calls geojson() itself right after writing the
file, so the (potentially tens-of-seconds, for a large POI export)
conversion cost is paid during the upload — covered by the frontend's
existing "Uploading…" status — instead of on whichever checkbox
toggle happens to be first, which otherwise reads as a silent hang
indistinguishable from broken. An unparseable file still saves fine;
the error just surfaces on first view same as before.
A several-thousand-waypoint park4night area export re-parses the
whole file with REXML on every toggle — ~2.4s for a 5k-waypoint/3.6MB
sample, so tens of seconds for a 50k-waypoint one, likely the actual
cause behind a "Could not load" report on a 41.5MB export. MapLibre
already clusters the waypoint layer client-side, so the bottleneck was
never rendering — cache the GeoJSON conversion alongside the source
file (invalidated by mtime) instead of a bigger bbox/tiling rework.
A park4night/OsmAnd POI export (e.g. a p4n2gpx area export) tags every
waypoint with a category and a per-category color/icon in its <extensions>
block, all of which was previously thrown away — every waypoint rendered as
the same plain red dot, unreadable once a file has a few thousand of them.
Maptool::Gpx now captures category/color/icon (waypoint_style), matched by
namespace URI rather than local element name: a p4n2gpx file carries both
<osmand:icon> and <locus:icon> side by side, and REXML strips prefixes from
Element#name, so both report as just "icon" — matching on that alone was
silently picking the (useless) Locus drawable path instead of the OsmAnd
category key. Regression test added for this.
On the map: waypoints are colored by category (falling back to the plain
track color for anything uncategorized), a small hand-drawn icon set
(public/js/icons.js's POI_ICONS, rasterized to SDF sprites at runtime) draws
a glyph on top for the categories it covers, the points source is clustered
so a dense export collapses into count bubbles instead of a solid blob, and
a legend in the side panel lists whichever categories are currently visible.
Splitting each file's GeoJSON into a lines source and a points source
(rather than one mixed source with a geometry-type filter per layer) was
needed for clustering, since MapLibre's cluster support only understands a
source made entirely of points.
GPX tracks were silently failing to render for large real-world files:
point_extensions used REXML's "extensions//*" XPath, and the // descendant
axis costs low-single-digit milliseconds per call regardless of subtree
size. Run once per point, that's negligible on small files but ~30s on a
real 8000-point track — long enough to hit a proxy timeout in production,
so the file just never appeared with no error. Replaced with plain
recursive child traversal (~40x faster on the same file, confirmed
29.9s -> 3.16s); test_parsing_many_points_with_extensions_stays_fast
guards against reintroducing it.
Also adds support for <rte> routes (not just <trk> tracks and <wpt>
waypoints), sharing the track/route conversion logic via a new
line_feature helper. Routes get their own "route" kind and a dashed
ROUTE_ICON in the file tree so the type is visible at a glance.
TrackStats.build_profile now also computes elapsed_secs per point
(relative to the first known timestamp), alongside the existing
cumulative distance. Point x-positions stay distance-based either
way — only the shared axis strip's 5 tick labels (and the tooltip)
switch between the two, so toggling is a cheap re-render rather than
rebuilding the charts. The toggle only appears when a track actually
has elapsed-time data to switch to.
Track properties gain activity_type (FIT: sport enum from the
session/sport message, Garmin's own table; GPX: <trk><type> passed
through as-is) alongside the existing distance/elevation/duration.
Clicking a track line now opens a bottom panel with a chart per
metric the track actually has (elevation always; speed/cadence/heart
rate only when present — FIT's record fields or GPX's device
extensions). TrackStats.build_profile computes cumulative distance
the same way for both formats so they're directly comparable. One
pointermove listener drives a synchronized crosshair across every
chart, a single tooltip listing all series, and a map marker at the
hovered point's position — hovering the chart shows where on the
track that moment was.
`bundle exec rake test` (minitest, a default gem but pinned in
Gemfile for reproducibility) now covers what ad-hoc scripts kept
re-verifying by hand during development: Gpx/Fit/TrackStats parsing,
GpxStore/LiveTrackStore file/TTL logic, LiveTrackClient's payload
parsing, and app.rb's routes end to end via Rack::MockRequest.
MAPTOOL_DATA_DIR (new, app.rb) lets the suite point GPX_STORE/
LIVETRACK_STORE at a throwaway directory instead of real data/ —
without it every run would leave uploaded test files behind. Each
lib file's old `if __FILE__ == $0` self-check is now redundant with
real tests covering the same ground, so removed rather than kept as
a second copy of the same assertions. Wired into ci.yml so it isn't
just a local convenience.
randy