maptool/docs/architecture.adoc

= Architecture
:toc:
:toclevels: 2

== Overview

MapTool is a Sinatra web application, served locally by Puma and
opened in a browser.

== Directory layout

[cols="1,3"]
|===
| Path | Purpose

| `app.rb`, `config.ru`
| Sinatra application and Rack entry point.

| `config/puma.rb`
| Puma server configuration (binds to `127.0.0.1` only — the app is
  local-only, not exposed on the network).

| `public/`, `views/`
| Frontend assets and Sinatra templates. The map view uses MapLibre GL
  JS, rendering vector tiles.

| `public/vendor/`
| Third-party frontend libraries, vendored (not loaded from a CDN) so
  the app also works with no internet access. See
  `docs/third-party-notices.adoc`.

| `data/`
| Local, gitignored data files, e.g. `offline.pmtiles` and uploaded
  GPX/FIT files under `data/gpx/`.

| `lib/maptool/`
| Application code beyond routing: the GPX/FIT parsers and file store.

| `docs/`
| Project documentation, in AsciiDoc.
|===

== Offline maps

Map data is vector, not raster: the app ships
https://maplibre.org[MapLibre GL JS] (vendored, see below) and the map
view (`public/js/map.js`) offers two styles:

* *Online* — a full hosted vector style,
  https://tiles.openfreemap.org/styles/liberty[OpenFreeMap "liberty"],
  requires internet access.
* *Offline* — built at runtime from a local
  https://github.com/protomaps/PMTiles[PMTiles] archive at
  `data/offline.pmtiles`, served as-is (see below) at
  `/tiles/offline.pmtiles`, decoded client-side by
  https://github.com/protomaps/PMTiles[pmtiles.js] registered as a
  `pmtiles://` protocol handler, and styled with base layers from
  https://github.com/protomaps/basemaps[protomaps-themes-base].

`GET /api/offline-tiles` reports whether `data/offline.pmtiles` is
present (`{"available": true|false}`); the frontend uses it once, on
load, to default to the offline style when local tile data exists —
otherwise it defaults to the online style. Either way, the layer
toggle in the top-left of the map lets the user switch manually.

PMTiles is a single, self-indexed file, which is why this needs no
tile-decoding logic on the server: `app.rb`'s `/tiles/offline.pmtiles`
route (`send_file`, which Sinatra serves with HTTP range-request
support) just returns byte ranges of the file; pmtiles.js in the
browser reads the archive's own directory and requests only the bytes
it needs for the tiles on screen.

=== Offline text labels and color palette

Label rendering in MapLibre needs a "glyphs" server serving font PBF
ranges (256 Unicode code points each); `data/offline.pmtiles`'s fields
(`name`, `name:en`, `script`, ...) match what
`protomaps-themes-base`'s `layers(source, theme, { lang: "en" })`
expects for its label layers (verified against the actual PMTiles
vector layer schema), so passing that option turns labels on. The
glyphs themselves are vendored under `public/vendor/fonts/` — Noto
Sans Regular/Italic/Medium, ranges `0-255` and `256-511`
(Basic Latin + Latin-1 Supplement + Latin Extended-A, SIL OFL
licensed, from
https://github.com/protomaps/basemaps-assets[protomaps/basemaps-assets]),
covering accented Latin place names (Slovak, Czech, Polish, German,
...). Labels in other scripts (Cyrillic, Greek, CJK, ...) won't render
offline, since those glyph ranges aren't vendored.

`public/js/map.js`'s `OFFLINE_THEME` also nudges a handful of color
roles (background, water, wood, buildings, the two road-importance
tiers) from `protomaps-themes-base`'s stock "light" theme towards the
online "liberty" style's actual palette (sampled from its style JSON),
so Online/Offline don't look like two different apps. This is
deliberately partial — most roles (labels, casings, minor terrain
fills, ...) are left at their theme defaults — not the kind of
full per-layer recolor a previous iteration of this app did (and
removed, along with any dependency on OsmAnd's own tile server, which
this app cannot use per OsmAnd's terms restricting tile use to "personal,
non-commercial use within OsmAnd applications and platforms").

No PMTiles file ships with the repository — it is expected to be
supplied locally (see `docs/development.adoc`) and stays gitignored
because such extracts are typically large binary files.

== Weather radar overlay

`public/js/radar.js` adds a "Radar" map control that overlays recent
precipitation radar from https://www.rainviewer.com[RainViewer]'s free
public API (no API key required). Enabling it:

. Fetches `https://api.rainviewer.com/public/weather-maps.json`,
  which lists RainViewer's past radar frames (fixed 10-minute
  cadence, at most 13 covering the last 120 minutes).
. Takes the most recent 7 (last ~60 minutes — the same default as
  the https://github.com/jpettitt/weather-radar-card[Home Assistant
  weather-radar-card], for a consistent feel with that setup) and
  adds one MapLibre raster source/layer per frame, all pre-loaded but
  only the current frame's `raster-opacity` non-zero.
. Animates by toggling that opacity across frames on an interval
  (play/pause + a scrub slider), rather than swapping tiles in and out
  — avoids reloading tiles on every step.

Tile URL: `{host}{frame.path}/256/{z}/{x}/{y}/2/1_0.png` — `2` is
RainViewer's "Universal Blue" color scheme, `1` enables their
smoothing, `0` disables the snow overlay. `host` and each frame's
`path` come from the API response.

Since `map.setStyle()` (switching Online/Offline) tears down all
custom sources and layers, the control listens for `style.load` and
re-adds the radar layers if the overlay was on.

This is a live, online-only overlay — it always needs internet access
and is unrelated to the offline PMTiles basemap above. Per RainViewer's
terms, the API is free for personal/non-commercial use.

== GPX/FIT tracks and waypoints

The "Tracks" map control (`public/js/gpx.js`) opens a side panel to
upload GPX or FIT files by dragging them in, organize them into
folders (also by dragging), and toggle each one on the map
individually.

Storage:: Files are stored as-is under `data/gpx/`, in a
folder hierarchy the user manages directly — `Maptool::GpxStore`
(`lib/maptool/gpx_store.rb`) provides `save`, `create_folder`,
`delete_file`, `delete_folder` and `move`. Every path coming from the
client (a folder to create/upload/move into, a file path in a URL) is
resolved with `File.expand_path` and checked against the store root
before touching the filesystem, so `../..`-style path traversal can't
escape `data/gpx/`; `delete_folder` additionally refuses an empty path
or one that resolves to the store root itself, so it can never `rm
-rf` the whole store.
`GET /api/gpx/tree` returns the resulting folder/file tree as JSON,
which the panel renders recursively (folders as `<details>`, files as
a checkbox, both with a delete button).

Drag & drop:: The dropzone at the top of the panel accepts one or more
`.gpx`/`.fit` files, or a `.zip` (`POST /api/gpx`, multipart) into the
top level, or click it to browse instead. A `.zip` — Garmin's bulk/FIT
export format — isn't stored itself: `save_zip_entries` (`app.rb`)
expands it with `rubyzip` and saves each `.gpx`/`.fit` entry it
contains individually (flattened — any folder structure inside the
zip is discarded), skipping anything else (readme files, etc.) it
might hold. An already-uploaded file (a
`.gpx-tree__file` row, `draggable`) can be dragged onto a folder to
move it there (`POST /api/gpx/move`), or dropped anywhere else in the
tree to move it back to the top level; an OS file can also be dropped
directly onto a folder to upload straight into it. Both drag sources
land on the same drop targets and are told apart by a custom
`application/x-gpx-path` drag payload type versus the browser's own
`"Files"` type. If a file that's currently checked (visible on the
map) gets moved, its map layers are re-added under the new path so it
stays visible; if it (or its containing folder) gets deleted, its
layers are removed instead.

Rendering:: Checking a file fetches `GET /api/gpx/file/<path>`, routed
by extension to `Maptool::Gpx` or `Maptool::Fit`
(`lib/maptool/gpx.rb`/`fit.rb`) for parsing, and both convert to the
same GeoJSON `FeatureCollection` shape so the frontend doesn't need to
know which format it got. `Gpx` parses the GPX with REXML: `<wpt>`
waypoints become `Point` features (with `name`, `desc` and `time`
carried through as properties), each `<trk>` becomes one
`MultiLineString` feature (one line per `<trkseg>`), and each `<rte>`
becomes one too (a route is a single continuous line by definition, no
`<trkseg>`-style grouping) — the two are told apart only by a `kind`
property (`"track"` vs `"route"`; waypoints are `"waypoint"`), which
`Gpx.kinds` also exposes per-file so the file tree can show a dashed
route icon next to a route file and a solid one for a track
(`public/js/icons.js`'s `ROUTE_ICON`/`TRACK_ICON`,
`public/js/gpx.js`'s `_kindIcon`). `<rte>` has no GPX equivalent of
`<trk>`'s `<type>`, so a route's `activity_type` is always `nil`.
`Fit` reads Garmin's binary FIT format directly
(hand-rolled — see below) into a single `MultiLineString` `track`
feature; FIT activity files don't really have a GPX-waypoint
equivalent, so it never produces `Point` features. Checking a file
fetches its GeoJSON itself (`public/js/gpx.js`'s `_addLayers`) and
hands the parsed result to `addSource` — rather than passing MapLibre a
URL and letting it fetch internally — specifically so a `422` (a file
REXML or `Fit` can't parse) surfaces as an alert and un-checks the box
instead of the layer just silently never appearing, which is what a
`data: url` source does on a failed fetch. The fetched features are
then split client-side into two GeoJSON sources per file, a
lines-and-routes one and a points-only one, rather than one mixed
source with a `geometry-type` filter per layer (the previous design):
MapLibre's built-in point clustering (below) only understands a source
made entirely of `Point`/`MultiPoint` features, so a file mixing a
track with waypoints — or a several-thousand-waypoint POI export with
no line at all — needs the points on their own source regardless of
what else the file's GeoJSON also carries.

POI category styling & clustering:: A POI export (park4night via
p4n2gpx, OsmAnd) tags each waypoint with a category (`<type>`) and, in
a vendor `<extensions>` block, a per-category color and OsmAnd icon key
(e.g. `amenity_parking`) — `Maptool::Gpx.waypoint_style` reads both
into `category`/`color` properties (an OsmAnd export can carry a
`<locus:icon>` alongside `<osmand:icon>`, both of which strip down to
the same local element name "icon" once REXML drops namespace
prefixes, so this specifically matches on namespace *URI*
(`https://osmand.net`), not local name, to avoid picking whichever
happens to come last in the file). Individual waypoints render as a
rounded-square badge (`public/js/icons.js`'s `MAP_MARKER_ICON`) rather
than a plain circle — a flat colored dot turned out to look too much
like a base map style's own small circular POI icons, blending in
instead of reading as "your data". A rounded square rather than a
pin/teardrop: it matches how the National Park Service itself frames
these exact category symbols (below) on its own maps, so the two look
native together instead of like mismatched icon styles. It's colored
via `["coalesce", ["get", "color"], TRACK_COLOR]` (falling back to the
plain track color for a waypoint with no category, e.g. a hand-placed
GNSS pin) and — unlike a pin, which has to anchor at its tip — centered
directly on the coordinate with plain default anchoring, which also
means the category-glyph layer on top needs no offset math: both
layers just share the same center anchor at `[0, 0]`. A small curated
icon set (`public/js/icons.js`'s `POI_ICONS`, covering the categories
p4n2gpx actually uses, not an exhaustive OsmAnd icon set) draws a white
glyph on top of the badge for any waypoint whose `icon` matches a known
key (filtered out entirely for anything else, so an unrecognized icon
key just falls back to a plain colored badge rather than a
broken/missing image warning). Six of the seven glyphs are the
National Park Service's own map symbols
(`github.com/nationalparkservice/symbol-library`, BSD-licensed,
originally forked from Mapbox's Maki project) — a professional,
purpose-built icon set beats hand-drawing "fuel pump"/"picnic table"
glyphs from scratch; `forest` has no NPS equivalent (that set covers
facilities, not land cover) and stays hand-drawn. Both the badge and
the category glyphs are rasterized once per style load
(`_registerPoiIcons`, an SVG → `<img>` → `<canvas>` → `ImageData` round
trip, since these need to be real raster images for `map.addImage`) and
registered as SDF images — a single alpha shape MapLibre tints at
render time via `icon-color` rather than a fixed-color image. The
points source itself
is created with
`cluster: true`; nearby points collapse into a stepped-radius,
sequential-blue circle sized by `point_count` below `clusterMaxZoom`,
clicking one zooms to its `getClusterExpansionZoom()`, and this is
harmless to enable unconditionally — a source with nothing dense enough
to cluster just renders every point individually as usual. A small
legend (`.gpx-legend` in the side panel) lists every distinct category
+ color currently visible across all checked files, rebuilt from the
same points GeoJSON kept around for exactly this
(`GpxPanelControl#_pointsByPath`) whenever a file is toggled on/off; it
stays hidden for files with no categorized waypoints (a plain GPS
track has none). The legend swatch color is set via
`swatch.style.background = color` (a DOM property assignment) rather
than interpolating the untrusted color string into an `innerHTML`
attribute — `escapeHtml` (used elsewhere on this page) only guards
element content, not attribute-quote escaping, so building
`style="background:${color}"` as a string would have been an
injection risk for a crafted GPX file.

FIT parsing:: No gem was added for this — the one Ruby option
(`fit-parser`, tjwallace/fit) hasn't been touched since 2022 and pulls
in `activesupport` for a single-purpose parser, a heavy, stale
dependency for what this app actually needs. `Maptool::Fit` instead
reads only "record" messages (global message number 20 — every other
message type, laps/sessions/events/device info/developer fields, is
skipped by byte length by summing each field's declared size, never
interpreted) for 3 fields: `position_lat`/`position_long` (field 0/1,
`sint32` semicircles, converted to degrees by `* (180 / 2**31)`) and
`altitude`/`enhanced_altitude` (field 2/78, scale 5 / offset 500 —
`raw / 5.0 - 500`, in meters). Field numbers and scale/offset are
Garmin's own (github.com/garmin/fit-python-sdk's `profile.py`), not
guessed. Both the message framing (headers, definition messages,
compressed-timestamp headers, little/big-endian) and the 3 fields'
values were checked against `github.com/dtcooper/python-fitparse`'s
own test fixtures — 5 real device files spanning 2013-2019, one with
developer fields, one using only compressed-timestamp headers — byte
after byte, all coordinates and altitudes exact, before this shipped.

`.zip` extraction does use a gem (`rubyzip`) — parsing the ZIP central
directory format correctly (including compressed entries) isn't the
kind of narrowly-scoped, single-purpose logic FIT parsing above is,
and rubyzip is the de facto standard for this in Ruby.

Like the radar overlay, GPX layers are removed by `map.setStyle()`
(switching Online/Offline) and get re-added on `style.load` for
whichever files were checked.

Waypoint popups:: Clicking a waypoint circle opens a MapLibre popup
with its name, category (if any), description and time. Click/hover handling is
registered once per file (via `map.on(type, layerId, handler)`, a
delegated listener that survives `setStyle()`) rather than on every
`style.load` re-add, which would otherwise stack duplicate handlers
and pop up several copies per click. Waypoint text comes straight from
an uploaded file — untrusted input — so `public/js/gpx.js`'s
`linkify()` turns bare URLs in the description into `<a>` links by
splitting the text on a URL pattern and escaping every piece
(including the URL itself, as both `href` and link text) before
assembling any HTML; the URL pattern excludes quote characters, so a
matched URL can never break out of the `href="..."` attribute. Every
other value is HTML-escaped before being handed to `Popup#setHTML`.

Track properties:: Every track feature (both formats) also carries
`distance_meters`, `elevation_gain_meters`, `duration_secs`,
`point_count` and `activity_type`, computed once server-side by
`Maptool::TrackStats` (`lib/maptool/track_stats.rb`, shared by
`Gpx`/`Fit` so neither duplicates the haversine formula) rather than
in the browser — cheap either way, but it means a GPX and a FIT file
end up with the same properties shape, so the frontend doesn't need to
know which format it got. Distance sums the haversine distance between
consecutive points; elevation gain sums only the positive deltas
(descents don't subtract); duration is the gap between the first and
last known timestamp, skipping unknown ones in between — `nil` for any
of these where the source data doesn't have enough to compute it (e.g.
a GPX file with no `<time>` elements has `duration_secs: nil`).
`activity_type` comes from wherever each format keeps it: FIT's
`session`/`sport` message has a `sport` enum field (`Fit::SPORT_NAMES`
— Garmin's own table, github.com/garmin/fit-python-sdk's `profile.py`,
not exhaustive of every value Garmin has defined but every mainstream
one); GPX's `<trk><type>` is free text, whatever a device/app chose to
write, passed through as-is. Hovering a track line (rather than
clicking, unlike a waypoint — these properties describe the whole
line, not whichever point the cursor happens to be over) shows them in
the same popup mechanism as waypoints; the popup also hints that
clicking opens the full profile panel below.

Track profile panel:: Clicking a track line opens a bottom panel
(`public/js/track_profile.js`, `TrackProfilePanel`) with a chart per
metric the track actually has data for — elevation always (assuming
any point carries an `<ele>`/altitude), speed/cadence/heart rate only
when present. `Maptool::TrackStats.build_profile` (called from both
`Gpx`/`Fit`, alongside `.compute` above) produces this: one entry per
point with cumulative distance (computed the same way for every
format, not trusted from a device's own odometer field, so a GPX and a
FIT track are directly comparable), elapsed time since the first known
timestamp, elevation, and whatever speed/cadence/heart-rate extras
that point's format/file carried — `nil` fields simply don't render
their chart (or, for `elapsed_secs`, don't offer the axis toggle
below). For FIT, those extras are the `record` message's own
`heart_rate`/`cadence`/`speed` fields (3/4/6); for GPX, a
device-specific `<extensions>` block under each `<trkpt>` — usually
Garmin's `TrackPointExtension`, matched by local element name only
(`hr`/`cad`/`speed`) so it doesn't care which XML namespace prefix a
given file happens to declare. That `<extensions>` block is walked by
hand (a plain recursive `elements.each`) rather than REXML's
`"extensions//*"` XPath, which this used originally: the `//`
descendant axis costs low-single-digit milliseconds *per call*
regardless of how small the subtree actually is, and this runs once
per point — negligible on a short file, but ~30s for a real
8000-point one (enough to hit a proxy timeout in production, so the
track just silently never rendered). Plain traversal is ~40x faster on
the same file; `test/test_gpx.rb`'s
`test_parsing_many_points_with_extensions_stays_fast` is a regression
test for it.
+
Each chart is its own single-series `<svg>` (elevation/speed/cadence/
heart rate are on wildly different scales, so this is small multiples
rather than one chart with several y-axes) with a fixed `viewBox` width, stretched to
the container by CSS; mouse interaction converts `clientX` through the
container's current `getBoundingClientRect()` on every move rather
than fighting the SVG's own coordinate transform, so it stays correct
across container width without redrawing — though a browser resize
while the panel is open won't reflow the already-drawn paths until
it's reopened. One `pointermove` listener on the whole chart container
(not per-chart) finds the nearest profile point by distance and drives
three things at once: a synchronized crosshair line in every chart, a
single tooltip listing every active series' value at that point, and a
MapLibre `Marker` moved to that point's `lon`/`lat` on the map — so
hovering the chart shows where on the track that moment actually was.
Each chart's own min/max are shown as plain HTML overlays in its
top-left/bottom-left corners (not SVG `<text>` — the viewBox's
non-uniform scaling would stretch it). Below the stack, a single
shared x-axis strip (again HTML, same reason) shows 5 evenly-spaced
ticks along the shared distance scale; a Distance/Time toggle above it
(only shown when the track has any elapsed-time data at all) switches
what those 5 ticks are labeled with — the underlying point positions
stay distance-based either way, only the tick text changes, so
toggling is a cheap re-render of just that strip.

== Live GNSS position

A "GNSS" map control (`public/js/gnss.js`) shows the fix from a local
GNSS receiver (a real one, plugged into the machine running the
server — distinct from `NavigationControl`'s browser
`navigator.geolocation`, which the OS may or may not derive from such
a receiver at all).

`lib/maptool/gpsd_client.rb` is a small client for
https://gpsd.io[gpsd]'s JSON protocol over TCP (port 2947 by default):
it watches for TPV — "time-position-velocity", gpsd's term for a fix —
reports and yields each as a plain hash, reconnecting on any I/O error
so a receiver that isn't plugged in yet (or gpsd not running) is not
fatal. `app.rb` runs one of these in a background thread for the
whole process and keeps the latest fix in `GNSS_LATEST`, guarded by a
mutex since the HTTP threads read it concurrently.

`GET /api/gnss/position`, polled by the frontend every 1.5s, is a
deliberate choice over a push channel (SSE/WebSocket): gpsd itself
only reports at roughly 1Hz, so a push channel buys nothing here, and
it sidesteps a real problem — Sinatra's `stream(:keep_open)` helper,
absent an EventMachine reactor (i.e. under plain threaded Puma, as
here), calls its block in a tight loop instead of waiting for external
writes. That is fine for a stream that itself blocks between writes,
but for a "register once, get pushed to later" connection — which is
what a live position feed is — the block returns immediately and gets
re-invoked immediately, forever: every iteration re-registers the
connection, so held connections and memory grow unbounded. This was
caught in testing (the dev server hit ~98% CPU and ~28GB RSS within
minutes) before it shipped.

The endpoint reports `available: false` once the last fix is more than
`GNSS_STALE_AFTER` (10s) old, rather than serving a stale position
forever after the receiver loses its fix or gets unplugged.

MapLibre `Marker`s (used for the fix) are plain positioned DOM
elements, not tied to the style's sources/layers, so — unlike the
radar/GPX layers — the marker survives `map.setStyle()` (switching
Online/Offline) with no extra re-add handling needed.

== Networking

By default (local dev) the Sinatra server only binds to `127.0.0.1`
and isn't reachable from other machines. The deployed instance is an
exception — see `docs/deployment.adoc` for `BIND_HOST` and how it's
exposed via Docker/nginx as `map.priesol.net`.

== Authentication

The GPX store's read endpoints (viewing the map, the folder tree,
individual files) are public — anyone with the URL can look. Its write
endpoints (`POST /api/gpx`, `POST /api/gpx/folder`, `DELETE
/api/gpx/file/*`, `DELETE /api/gpx/folder/*`, `POST /api/gpx/move`)
require login, guarded by `require_login!` in `app.rb`.

Login delegates to the Gitea instance at `git.priesol.net` via OAuth2
(`GET /login` redirects to Gitea's authorize endpoint, `GET
/auth/callback` exchanges the code and fetches
`/api/v1/user`) rather than maintaining a separate local
username/password — there's already an account to reuse, and it keeps
credential storage (password hash, resets, ...) out of this app
entirely. Successfully authenticating with Gitea is not by itself
enough to write, though — the callback also checks the returned
`login` against `MAPTOOL_ALLOWED_GITEA_USER` (a comma-separated
allowlist, e.g. `randy,friend1,friend2`), so registering (or already
having) any other account on that Gitea instance doesn't grant access
here. It's a flat allowlist — everyone in it has equal write access,
no per-user roles or permissions.

Session state is a signed cookie (`Rack::Session::Cookie`, `secure`
in production, `httponly`, `same_site: lax`) keyed off
`SESSION_SECRET` — see `docs/deployment.adoc` for where that and the
OAuth2 client id/secret live (not committed to the repo).