maptool/docs/development.adoc

Development guide

Table of Contents

• Web app only
• Offline map data
• Offline terrain data (hillshade)
• In-app regeneration
• Camper-relevant POIs (Overpass)
• Import plugins
• Offline Wikipedia (kiwix-serve)
• GNSS position (gpsd)
• Tests
• Continuous integration

Web app only

bundle install
bundle exec puma -C config/puma.rb

Open http://127.0.0.1:4567. For auto-reload during development, run via the rerun gem instead:

bundle exec rerun -- bundle exec puma -C config/puma.rb

Tip	Browsers apply heuristic caching to static files even without explicit cache headers, so an edit to public/ (or views/) can appear not to have taken effect on the next reload. Force a real reload (e.g. Ctrl/Cmd+Shift+R) if that happens.

Offline map data

To enable the offline style, place a PMTiles file at data/offline.pmtiles (see docs/architecture.adoc for how it is served and styled). Logged-in users can generate this from the app itself — see "In-app regeneration" below — or it can be built by hand:

• Extract a region from Protomaps' own daily full-planet basemap build (build index, files at https://build.protomaps.com/<key>;, ~120GB, ODbL/OSM attribution) with the go-pmtiles CLI’s extract subcommand, which pulls only the byte ranges covering a bbox over HTTP range requests rather than the whole file: pmtiles extract https://build.protomaps.com/<key>; data/offline.pmtiles --bbox=west,south,east,north --maxzoom=15. (The old app.protomaps.com extract-builder UI this section used to point at has since redirected elsewhere — this is the same underlying mechanism, just the CLI instead of a web UI.)

• To convert some other vector tile source (e.g. an .mbtiles file) into PMTiles instead, use the same CLI’s convert subcommand: pmtiles convert in.mbtiles data/offline.pmtiles. Its schema is what protomaps-themes-base (the styling library used for the offline style) expects.

Once the file is in place, GET /api/offline-tiles reports available: true and the map defaults to the offline style.

Offline terrain data (hillshade)

The sidebar’s "Terrain shading" toggle works online out of the box (public Terrarium elevation tiles, no API key). For it to work offline too, place a PMTiles archive of Terrarium-encoded raster-dem tiles at data/terrain.pmtiles (GET /api/terrain-tiles reports available: true once it’s there, same pattern as offline map data above). Also generatable in-app (see below) or by hand, the same go-pmtiles extract mechanism as data/offline.pmtiles above — but from Mapterhorn, a Protomaps-affiliated project publishing pre-built Terrarium-encoded terrain as PMTiles (a global planet.pmtiles, z0-12, plus ~450 regional chunks covering z13 and up — see their own data-access page for the full manifest and per-source license/attribution details, CC-BY-4.0 and similar, varying by region):

pmtiles extract https://download.mapterhorn.com/planet.pmtiles planet-extract.pmtiles --bbox=west,south,east,north --maxzoom=15
# only if a regional chunk covers the bbox too (see download_urls.json) —
# most bboxes hit exactly one, a bbox straddling a chunk boundary hits two:
pmtiles extract https://download.mapterhorn.com/<chunk>.pmtiles chunk-extract.pmtiles --bbox=west,south,east,north --maxzoom=15
pmtiles merge planet-extract.pmtiles chunk-extract.pmtiles data/terrain.pmtiles

Tiles are 512px WebP, not the 256px PNG this app’s online Terrarium source uses — `public/js/dem.js’s own point-elevation lookups (`shadow.js’s terrain-shadow overlay) read the actual decoded tile size rather than assuming one, so this isn’t a problem either way.

In-app regeneration

A logged-in user can also generate both files above from the app itself, no local tooling needed — the sidebar’s "Offline data" icon (Offline hub group) shows a checklist of European countries (lib/maptool/data/europe_countries.geojson, Natural Earth 110m admin-0, public domain) plus each file’s status (size, last updated). Ticking countries and clicking "Apply changes" saves the selection (data/offline_regions.json, an array of ISO A2 codes) and rebuilds both files to cover it.

There’s no standalone per-country file to individually delete — offline.pmtiles/terrain.pmtiles are always one combined extract for the whole current selection. "Remove a country" and "refresh a country" are both just "change the selection, then Apply" (unticking a country and applying rebuilds without it; nothing to separately delete). Every Apply re-extracts the entire selection from scratch, not just what changed — fine given extraction is fast (byte-range requests against remote archives, not per-tile fetches), but worth revisiting if someone selects half of Europe at once and finds it slow.

This wraps exactly the manual go-pmtiles workflows documented above (lib/maptool/pmtiles_builder.rb) — the server needs the go-pmtiles CLI installed and on PATH (override with the PMTILES_BIN env var if it lives somewhere else), using --region=<geojson> (a FeatureCollection of the selected countries — go-pmtiles treats multiple features as their union already, no union/dissolve math needed on this side) rather than --bbox= for precise, per-country extraction. terrain.pmtiles extracts planet.pmtiles (always) plus whichever Mapterhorn regional chunks intersect the selection, merge`s them if there was more than one — always safe regardless of how many/which countries are selected, since each chunk is Mapterhorn’s own pre-partitioned, non-overlapping source file (extracting the same source once per country and merging those, by contrast, would fail — adjacent countries' region-extracts share border tiles, since extraction copies whole tiles rather than clipping vector content to the polygon, and `pmtiles merge refuses any overlapping tile id between inputs).

A build runs in the background (GET /api/offline-regions to list/see the current selection, POST to change it, then POST /api/offline-tiles/build / /api/terrain-tiles/build, polled via the matching */build-status route) — only one of each kind at a time; starting a second while one’s running gets a 409, as does starting either with nothing selected yet.

Camper-relevant POIs (Overpass)

rake oversouth,west,north,east,out.geojson queries Overpass for drinking water, dump stations, LPG refill, public toilets, laundromats, supermarkets/convenience stores, and springs within a bbox, and writes the result as a .geojson file (lib/maptool/overpass_pois.rb). Nodes only (nothing that’s mapped as a way/relation, e.g. a supermarket traced as a building outline rather than a single point). Also reachable from inside the running app itself, without the command line — see Import plugins below, which wraps this exact query as plugins/overpass-pois.

Import plugins

The sidebar’s import icon (logged in only) lists whatever’s under plugins/ and runs one for the current map viewport with a click — no area-picker to fill in, the visible area is the area. Each plugin is a directory with a plugin.json manifest ({"label": "…​", "command": "./run.sh"}) plus whatever script that command names; the entire interface a plugin needs to know about maptool is "read --south/--west/--north/--east, print a GeoJSON FeatureCollection to stdout" (lib/maptool/plugins.rb) — the result is saved as a new .geojson file the same way an upload would be.

plugins/overpass-pois ships as the reference example (a thin wrapper around the rake overpass:pois logic above). A plugin that scrapes a site whose terms of service forbid that — Park4Night, iOverlander, Freecampsites — doesn’t have to live in this (public) repo to work here: drop its directory under plugins/ locally and just don’t git add it, same as any other local-only file this app already treats that way (data/offline.pmtiles, the kiwix ZIM, …​).

Offline Wikipedia (kiwix-serve)

A point popup’s "Nearby on Wikipedia" link needs kiwix-serve running with a Wikipedia ZIM loaded — set KIWIX_URL to wherever the server can reach it (docker-compose.yaml already runs a kiwix service pointed at data/kiwix/wikipedia.zim and wires KIWIX_URL to it via its compose service name; blank/unset disables the link, same as every other optional integration here). Used only server-side: production sits behind a reverse proxy that forwards only to this app, so there’s no address a browser could reach kiwix-serve at directly — GET /kiwix/* (app.rb) proxies to it instead, and the kiwix service runs with --urlRootLocation=/kiwix so its own emitted links/assets already carry that same prefix. Not part of the repo — a ZIM is a large, self-contained archive, obtain one locally:

• Kiwix’s own library has pre-built Wikipedia ZIMs — a _maxi variant per language/region includes images, _nopic is much smaller if browsing images offline doesn’t matter here.

• Download it to data/kiwix/wikipedia.zim (the exact filename docker-compose.yaml’s `kiwix service expects — rename after downloading, Kiwix’s own filenames are much more specific). A symlink there works too (ln -s /wherever/the-real-file.zim data/kiwix/wikipedia.zim), so the real file doesn’t have to sit inside this repo’s checkout at all — the compose file’s volume binds that exact path, not the data/kiwix/ directory, specifically so this resolves correctly (a symlink inside an already-mounted directory wouldn’t: the container would have to resolve it itself, in its own filesystem, where the host path it points to doesn’t exist).

Once it’s there, docker compose up -d kiwix (or a full restart) picks it up — GET /api/kiwix reports available: true and the "Nearby on Wikipedia" link appears, pointed at kiwix-serve’s own geo search (GET /kiwix/search?pattern=&latitude=&longitude=&distance=) through this app’s own proxy rather than at kiwix-serve directly — it already has a full search UI of its own, this just makes it reachable through the same origin/reverse proxy as the rest of the app.

GNSS position (gpsd)

The "GNSS" control needs gpsd running and pointed at a receiver (gpsd /dev/ttyUSB0, or whatever device the receiver shows up as). On openSUSE, gpsd is a package with systemd units (gpsd.service / gpsd.socket); on other distributions, check your package manager. No app-side configuration is needed — it just connects to 127.0.0.1:2947, gpsd’s default port.

To test without hardware or gpsd itself installed/running, run bin/gnss_simulator.rb — it speaks gpsd’s own JSON wire protocol directly on gpsd’s default port (127.0.0.1:2947), so Maptool::GpsdClient connects to it exactly as it would to real gpsd, no app-side changes needed. Drives a loop around a center point (Bratislava by default) at a driving-ish pace instead of sitting at a fixed position, since a stationary "GNSS position" isn’t much to look at on the map:

bin/gnss_simulator.rb                       # loop around Bratislava, 300m radius
bin/gnss_simulator.rb 48.85 2.35 500         # custom center/radius (Paris, 500m)

GET /api/gnss/position should report a fix within a couple of seconds, moving on every poll.

To test against gpsd itself instead (e.g. to exercise the actual NMEA-parsing path, not just gpsd’s JSON output), feed it a fake NMEA stream over TCP instead of a real serial device — gpsd accepts a tcp://host:port source:

# terminal 1: a TCP server repeating one NMEA GGA/RMC sentence pair
python3 - <<'PY' 5001
import socket, sys, time

def checksum(s):
    cs = 0
    for ch in s:
        cs ^= ord(ch)
    return f"{cs:02X}"

def sentence(body):
    return f"${body}*{checksum(body)}\r\n"

port = int(sys.argv[1])
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("127.0.0.1", port))
srv.listen(1)
conn, _ = srv.accept()
gga = "GPGGA,123519,4808.916,N,01706.462,E,1,08,0.9,140.4,M,46.9,M,,"
rmc = "GPRMC,123519,A,4808.916,N,01706.462,E,000.5,084.4,230824,003.1,W"
while True:
    conn.sendall((sentence(gga) + sentence(rmc)).encode())
    time.sleep(1)
PY

# terminal 2: point gpsd at it instead of a real device
gpsd -N -n tcp://127.0.0.1:5001

GET /api/gnss/position should then report the fake fix (Bratislava Castle, in the sentences above) within a couple of seconds.

Tests

bundle exec rake test runs the whole suite (test/test_*.rb, Minitest — a default gem, but pinned in the Gemfile’s `:test group for reproducibility, same reasoning as rake itself). test_app.rb drives app.rb’s routes end to end through `Rack::MockRequest rather than a real server; everything else tests one lib/maptool/ module in isolation. MAPTOOL_DATA_DIR (see app.rb) is pointed at a throwaway Dir.mktmpdir by test/test_helper.rb before app.rb loads, so running the suite never touches real data/ — the app-level tests also wipe that directory between each test (see AppTestHelper#teardown), since GPX_STORE/LIVETRACK_STORE are process-wide constants shared by every test in the run.

LiveTrackClient’s own tests use a synthetic fixture (no network) — its parsing was checked against real Garmin sessions by hand during development, not by the suite, since that needs a live, unexpired session token to fetch. `Fit’s tests likewise build a minimal FIT file by hand rather than shipping a real device fixture; correctness against real files (position/altitude fields, definition-message framing, compressed-timestamp headers) was checked by hand against `python-fitparse’s test fixtures during development — see `docs/architecture.adoc.

Continuous integration

.gitea/workflows/ci.yml runs on every push/PR against a self-hosted Gitea Actions runner: ruby -c on every .rb file, a require_relative "app" smoke check that the app loads without error, bundle exec rake test (see above), and node --check on every file under public/js/ — no JS test suite, just a syntax check.

.gitea/workflows/renovate.yml runs Renovate (the renovate/renovate image) weekly (Mondays, also triggerable manually), opening PRs for outdated dependencies (Bundler Gemfile/Gemfile.lock, auto-detected — no renovate.json needed) against this Gitea instance (RENOVATE_PLATFORM=gitea). It runs on the native deploy runner (see docs/deployment.adoc) as a one-shot docker run, not a persistent service. Needs a RENOVATE_TOKEN repo secret — a Gitea personal access token (Settings → Applications → Generate New Token, with repository read/write scope) from whichever account should author the PRs; RENOVATE_ONBOARDING=false skips Renovate’s usual first-PR "configure me" step and goes straight to dependency update PRs using its defaults.