= Development guide :toc: :toclevels: 2 == Web app only [source,shell] ---- bundle install bundle exec puma -C config/puma.rb ---- Open http://127.0.0.1:4567. For auto-reload during development, run via the `rerun` gem instead: [source,shell] ---- bundle exec rerun -- bundle exec puma -C config/puma.rb ---- TIP: Browsers apply heuristic caching to static files even without explicit cache headers, so an edit to `public/` (or `views/`) can appear not to have taken effect on the next reload. Force a real reload (e.g. Ctrl/Cmd+Shift+R) if that happens. == Offline map data To enable the offline style, place a PMTiles file at `data/offline.pmtiles` (see `docs/architecture.adoc` for how it is served and styled — the same OpenMapTiles schema/style the online map uses, just from local data). Logged-in users can generate this from the app itself — see "In-app regeneration" below — or it can be built by hand: * Download one or more country extracts from https://download.geofabrik.de[Geofabrik] (`.osm.pbf`, ODbL/OSM attribution) — e.g. `curl -O https://download.geofabrik.de/europe/slovakia-latest.osm.pbf`. * If more than one country, merge them with https://github.com/osmcode/osmium-tool[osmium-tool]: `osmium merge slovakia-latest.osm.pbf austria-latest.osm.pbf -o merged.osm.pbf` (each Geofabrik file is already clipped to that country's real border, so no separate bbox/polygon clip step is needed either before or after — skip this step entirely for a single country). * Tile it with https://github.com/onthegomap/planetiler[Planetiler]'s default profile, which *is* OpenMapTiles — no `--profile` flag needed — writing PMTiles directly: `java -jar planetiler.jar --osm-path=merged.osm.pbf --output=data/offline.pmtiles` (needs Java 21+; `planetiler.jar` from https://github.com/onthegomap/planetiler/releases/latest[the latest release]). Once the file is in place, `GET /api/offline-tiles` reports `available: true` and the map defaults to the offline style. == Offline terrain data (hillshade) The sidebar's "Terrain shading" toggle works online out of the box (public https://github.com/tilezen/joerd[Terrarium] elevation tiles, no API key). For it to work offline too, place a PMTiles archive of Terrarium-encoded `raster-dem` tiles at `data/terrain.pmtiles` (`GET /api/terrain-tiles` reports `available: true` once it's there, same pattern as offline map data above). Also generatable in-app (see below) or by hand, the same `go-pmtiles extract` mechanism as `data/offline.pmtiles` above — but from https://mapterhorn.com[Mapterhorn], a Protomaps-affiliated project publishing pre-built Terrarium-encoded terrain as PMTiles (a global `planet.pmtiles`, z0-12, plus ~450 regional chunks covering z13 and up — see https://mapterhorn.com/data-access/[their own data-access page] for the full manifest and per-source license/attribution details, CC-BY-4.0 and similar, varying by region): [source,shell] ---- pmtiles extract https://download.mapterhorn.com/planet.pmtiles planet-extract.pmtiles --bbox=west,south,east,north --maxzoom=15 # only if a regional chunk covers the bbox too (see download_urls.json) — # most bboxes hit exactly one, a bbox straddling a chunk boundary hits two: pmtiles extract https://download.mapterhorn.com/.pmtiles chunk-extract.pmtiles --bbox=west,south,east,north --maxzoom=15 pmtiles merge planet-extract.pmtiles chunk-extract.pmtiles data/terrain.pmtiles ---- Tiles are 512px WebP, not the 256px PNG this app's *online* Terrarium source uses — `public/js/dem.js`'s own point-elevation lookups (`shadow.js`'s terrain-shadow overlay) read the actual decoded tile size rather than assuming one, so this isn't a problem either way. == In-app regeneration A logged-in user can also generate both files above from the app itself, no local tooling needed — the sidebar's "Offline data" icon (Offline hub group) shows a checklist of European countries (`lib/maptool/data/europe_countries.geojson`, Natural Earth 110m admin-0, public domain) plus each file's status (size, last updated). Ticking countries and clicking "Apply changes" saves the selection (`data/offline_regions.json`, an array of ISO A2 codes) and rebuilds both files to cover it. There's no standalone per-country file to individually delete — `offline.pmtiles`/`terrain.pmtiles` are always one combined extract for the *whole current selection*. "Remove a country" and "refresh a country" are both just "change the selection, then Apply" (unticking a country and applying rebuilds without it; nothing to separately delete). terrain.pmtiles re-extracts its selection from scratch on every Apply — fine given that extraction is byte-range requests against a remote archive, not full downloads. offline.pmtiles, by contrast, downloads a whole country's `.osm.pbf` from Geofabrik (can be multi-GB), so it keeps a per-country cache under `data/geofabrik_cache/` instead of re-fetching on every Apply — a country already cached is only re-downloaded if Geofabrik's own `Last-Modified` for it has moved on since (checked with a cheap `HEAD`, see `PmtilesBuilder.cached_extract`), so growing or shrinking the selection only touches whatever's actually new to it. This wraps exactly the manual workflows documented above (`lib/maptool/pmtiles_builder.rb`). offline.pmtiles downloads each selected country's Geofabrik extract, `osmium merge`s them if there's more than one, then tiles the result with Planetiler — the server needs `osmium` and `java` + `planetiler.jar` installed and on `PATH` (override with the `OSMIUM_BIN`/`JAVA_BIN`/`PLANETILER_JAR` env vars if they live somewhere else; the Docker image installs all three, see `Dockerfile`). terrain.pmtiles still uses `go-pmtiles` (`PMTILES_BIN`) against Mapterhorn, unrelated to this — it extracts `planet.pmtiles` (always) plus whichever Mapterhorn regional chunks intersect the selection, `merge`s them if there was more than one — always safe regardless of how many/which countries are selected, since each chunk is Mapterhorn's own pre-partitioned, non-overlapping source file (extracting the *same* source once per country and merging those, by contrast, would fail — adjacent countries' region-extracts share border tiles, since extraction copies whole tiles rather than clipping vector content to the polygon, and `pmtiles merge` refuses any overlapping tile id between inputs). A build runs in the background (`GET /api/offline-regions` to list/see the current selection, `POST` to change it, then `POST /api/offline-tiles/build` / `/api/terrain-tiles/build`, polled via the matching `*/build-status` route) — only one of each kind at a time; starting a second while one's running gets a 409, as does starting either with nothing selected yet. Each successful build also writes a small sidecar (`offline.pmtiles. meta.json` — each source country's code/Geofabrik slug/Last-Modified; `terrain.pmtiles.meta.json` — the Mapterhorn source names/md5sums) recording what was actually built, since the `.pmtiles` file itself carries no such record. `GET /api/offline-tiles`/`/api/terrain-tiles` include it as `meta`, shown in the panel next to the existing size/updated line. `GET .../check-update` compares that recorded build against each source's live upstream state (a Geofabrik `HEAD` per country for offline, the Mapterhorn manifest for terrain — without rebuilding anything) and reports `update_available: true/false/null` — `null` means "don't know" (no local build yet, or a check itself failed), kept as its own route rather than folded into the plain status endpoints above since those are on the page-load hot path (the topbar's offline-status dot) and this one reaches an external service, only worth doing when the Offline data panel is actually open. == Camper-relevant POIs (Overpass) `rake overpass:pois[south,west,north,east,out.geojson]` queries https://overpass-api.de[Overpass] for drinking water, dump stations, LPG refill, public toilets, laundromats, supermarkets/convenience stores, and springs within a bbox, and writes the result as a `.geojson` file (`lib/maptool/overpass_pois.rb`). Nodes only (nothing that's mapped as a way/relation, e.g. a supermarket traced as a building outline rather than a single point). Also reachable from inside the running app itself, without the command line — see Import plugins below, which wraps this exact query as `plugins/overpass-pois`. == Import plugins The sidebar's import icon (logged in only) lists whatever's under `plugins/` and runs one for the current map viewport with a click — no area-picker to fill in, the visible area *is* the area. Each plugin is a directory with a `plugin.json` manifest (`{"label": "...", "command": "./run.sh"}`) plus whatever script that command names; the entire interface a plugin needs to know about maptool is "read `--south`/`--west`/`--north`/`--east`, print a GeoJSON `FeatureCollection` to stdout" (`lib/maptool/plugins.rb`) — the result is saved as a new `.geojson` file the same way an upload would be. `plugins/overpass-pois` ships as the reference example (a thin wrapper around the `rake overpass:pois` logic above). A plugin that scrapes a site whose terms of service forbid that — Park4Night, iOverlander, Freecampsites — doesn't have to live in this (public) repo to work here: drop its directory under `plugins/` locally and just don't `git add` it, same as any other local-only file this app already treats that way (`data/offline.pmtiles`, the kiwix ZIM, ...). == Offline Wikipedia (kiwix-serve) A point popup's "Nearby on Wikipedia" link needs https://github.com/kiwix/kiwix-tools[kiwix-serve] running with a Wikipedia https://library.kiwix.org[ZIM] loaded — set `KIWIX_URL` to wherever *the server* can reach it (`docker-compose.yaml` already runs a `kiwix` service pointed at `data/kiwix/wikipedia.zim` and wires `KIWIX_URL` to it via its compose service name; blank/unset disables the link, same as every other optional integration here). Used only server-side: production sits behind a reverse proxy that forwards only to this app, so there's no address a browser could reach kiwix-serve at directly — `GET /kiwix/*` (`app.rb`) proxies to it instead, and the `kiwix` service runs with `--urlRootLocation=/kiwix` so its own emitted links/assets already carry that same prefix. Not part of the repo — a ZIM is a large, self-contained archive, obtain one locally: * https://library.kiwix.org/#lang=eng&category=wikipedia[Kiwix's own library] has pre-built Wikipedia ZIMs — a `_maxi` variant per language/region includes images, `_nopic` is much smaller if browsing images offline doesn't matter here. * Download it to `data/kiwix/wikipedia.zim` (the exact filename `docker-compose.yaml`'s `kiwix` service expects — rename after downloading, Kiwix's own filenames are much more specific). A symlink there works too (`ln -s /wherever/the-real-file.zim data/kiwix/wikipedia.zim`), so the real file doesn't have to sit inside this repo's checkout at all — the compose file's volume binds that exact path, not the `data/kiwix/` directory, specifically so this resolves correctly (a symlink *inside* an already-mounted directory wouldn't: the container would have to resolve it itself, in its own filesystem, where the host path it points to doesn't exist). Once it's there, `docker compose up -d kiwix` (or a full restart) picks it up — `GET /api/kiwix` reports `available: true` and the "Nearby on Wikipedia" link appears, pointed at kiwix-serve's own geo search (`GET /kiwix/search?pattern=&latitude=&longitude=&distance=`) through this app's own proxy rather than at kiwix-serve directly — it already has a full search UI of its own, this just makes it reachable through the same origin/reverse proxy as the rest of the app. == GNSS position (gpsd) The "GNSS" control needs https://gpsd.io[gpsd] running and pointed at a receiver (`gpsd /dev/ttyUSB0`, or whatever device the receiver shows up as). On openSUSE, `gpsd` is a package with systemd units (`gpsd.service` / `gpsd.socket`); on other distributions, check your package manager. No app-side configuration is needed — it just connects to `127.0.0.1:2947`, gpsd's default port. To test without hardware or gpsd itself installed/running, run `bin/gnss_simulator.rb` — it speaks gpsd's own JSON wire protocol directly on gpsd's default port (`127.0.0.1:2947`), so `Maptool::GpsdClient` connects to it exactly as it would to real gpsd, no app-side changes needed. Drives a loop around a center point (Bratislava by default) at a driving-ish pace instead of sitting at a fixed position, since a stationary "GNSS position" isn't much to look at on the map: [source,shell] ---- bin/gnss_simulator.rb # loop around Bratislava, 300m radius bin/gnss_simulator.rb 48.85 2.35 500 # custom center/radius (Paris, 500m) ---- `GET /api/gnss/position` should report a fix within a couple of seconds, moving on every poll. To test against gpsd itself instead (e.g. to exercise the actual NMEA-parsing path, not just gpsd's JSON output), feed it a fake NMEA stream over TCP instead of a real serial device — gpsd accepts a `tcp://host:port` source: [source,shell] ---- # terminal 1: a TCP server repeating one NMEA GGA/RMC sentence pair python3 - <<'PY' 5001 import socket, sys, time def checksum(s): cs = 0 for ch in s: cs ^= ord(ch) return f"{cs:02X}" def sentence(body): return f"${body}*{checksum(body)}\r\n" port = int(sys.argv[1]) srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) srv.bind(("127.0.0.1", port)) srv.listen(1) conn, _ = srv.accept() gga = "GPGGA,123519,4808.916,N,01706.462,E,1,08,0.9,140.4,M,46.9,M,," rmc = "GPRMC,123519,A,4808.916,N,01706.462,E,000.5,084.4,230824,003.1,W" while True: conn.sendall((sentence(gga) + sentence(rmc)).encode()) time.sleep(1) PY # terminal 2: point gpsd at it instead of a real device gpsd -N -n tcp://127.0.0.1:5001 ---- `GET /api/gnss/position` should then report the fake fix (Bratislava Castle, in the sentences above) within a couple of seconds. == Tests `bundle exec rake test` runs the whole suite (`test/test_*.rb`, Minitest — a default gem, but pinned in the `Gemfile`'s `:test` group for reproducibility, same reasoning as `rake` itself). `test_app.rb` drives `app.rb`'s routes end to end through `Rack::MockRequest` rather than a real server; everything else tests one `lib/maptool/` module in isolation. `MAPTOOL_DATA_DIR` (see `app.rb`) is pointed at a throwaway `Dir.mktmpdir` by `test/test_helper.rb` before `app.rb` loads, so running the suite never touches real `data/` — the app-level tests also wipe that directory between each test (see `AppTestHelper#teardown`), since `GPX_STORE`/`LIVETRACK_STORE` are process-wide constants shared by every test in the run. `LiveTrackClient`'s own tests use a synthetic fixture (no network) — its parsing was checked against real Garmin sessions by hand during development, not by the suite, since that needs a live, unexpired session token to fetch. `Fit`'s tests likewise build a minimal FIT file by hand rather than shipping a real device fixture; correctness against real files (position/altitude fields, definition-message framing, compressed-timestamp headers) was checked by hand against `python-fitparse`'s test fixtures during development — see `docs/architecture.adoc`. == Continuous integration `.gitea/workflows/ci.yml` runs on every push/PR against a self-hosted Gitea Actions runner: `ruby -c` on every `.rb` file, a `require_relative "app"` smoke check that the app loads without error, `bundle exec rake test` (see above), and `node --check` on every file under `public/js/` — no JS test suite, just a syntax check. `.gitea/workflows/renovate.yml` runs https://docs.renovatebot.com[Renovate] (the `renovate/renovate` image) weekly (Mondays, also triggerable manually), opening PRs for outdated dependencies (Bundler `Gemfile`/`Gemfile.lock`, auto-detected — no `renovate.json` needed) against this Gitea instance (`RENOVATE_PLATFORM=gitea`). It runs on the native `deploy` runner (see `docs/deployment.adoc`) as a one-shot `docker run`, not a persistent service. Needs a `RENOVATE_TOKEN` repo secret — a Gitea personal access token (*Settings → Applications → Generate New Token*, with repository read/write scope) from whichever account should author the PRs; `RENOVATE_ONBOARDING=false` skips Renovate's usual first-PR "configure me" step and goes straight to dependency update PRs using its defaults.