Files
maptool/docs/deployment.adoc
T
randy 24ac5931f8
CI / check (push) Successful in 40s
CI / deploy (push) Successful in 12s
Link POI popups to offline Wikipedia via kiwix-serve
lib/maptool/kiwix.rb: a small reachability check for an external
kiwix-serve instance. GET /api/kiwix reports it (same optional-
integration pattern as offline-tiles/terrain-tiles); the frontend
adds a "Nearby on Wikipedia" link to a point's popup pointing straight
at kiwix-serve's own geo search (GET /search?latitude=&longitude=
&distance=, confirmed from libkiwix's actual source rather than
guessed) — linked to directly, not proxied/embedded, since it already
has a full search UI of its own.

docker-compose.yaml: new `kiwix` service (ghcr.io/kiwix/kiwix-serve)
on the default network, KIWIX_URL wired to reach it by service name.
The ZIM itself isn't part of the repo (large, self-contained archive)
— data/kiwix/ gitignored same as the *.pmtiles files, setup steps in
docs/development.adoc.

No kiwix-serve/ZIM available here to test end-to-end — verified the
app side against a stand-in local HTTP server (GET /api/kiwix flips
available correctly) and confirmed the generated popup link matches
the real geo-search query shape. docker-compose.yaml validated with
`docker-compose config`. Full test suite: 118 runs, 0 failures.
2026-09-09 13:44:37 +02:00

259 lines
13 KiB
Plaintext

= Deployment
:toc:
:toclevels: 2
MapTool deploys as a Podman container (via `podman compose`, driven by a
`podman-compose@.service` systemd template — see below) on the same server
the Gitea Actions runners run on. Two separate runners are registered
against the Gitea instance:
* The existing Docker-based runner (`gitea/runner:3`), labels
`ubuntu-latest`/`ubuntu-24.04`/`ubuntu-22.04` — runs the `check` job
in a fresh, isolated container per job, same as before.
* A dedicated native runner, installed directly on the host (no Docker
around it), label `cube-host` — runs only the `deploy` job.
They're separate because the Docker-based runner's jobs (whether in
its usual isolated per-job containers, or tried directly inside its
own container via a `host`-scheme label) have no reliable access to
the host's filesystem, `systemctl`, or even a full toolset (its own
container image is missing `rsync`, `docker` CLI, Node.js — it's built
for spawning per-job containers via its `docker.sock` mount, not for
running arbitrary host commands). The native runner sidesteps all of
that: it *is* a host process, so `docker`, `git`, etc. are just
whatever's installed on the server.
== Layout on the server
`/srv/podman/maptool/` (`/srv/podman` is a symlink to `/srv/docker`, a
leftover from before the Docker → Podman migration — kept around so
existing paths/tooling that still assume `/srv/docker` keep working;
expect this to flip the other way, `/srv/docker` symlinked to
`/srv/podman`, once nothing depends on the old name anymore) holds:
* A checkout of this repository (kept in sync by the deploy job, see
below) — `Dockerfile`, `docker-compose.yaml`, application code.
`podman compose` reads the same `docker-compose.yaml` — no
Podman-specific compose file needed.
* `data/`, *not* part of that sync — uploaded GPX files,
`offline.pmtiles`, `terrain.pmtiles`, and `kiwix/wikipedia.zim` (see
`docs/development.adoc` for each). Persisted via a bind mount
(`./data:/app/data` in `docker-compose.yaml`), so a redeploy never
touches it.
== The native deploy runner
Installed from the official binary (see
https://docs.gitea.com/runner/installation/binary/), running as a
systemd service (`gitea-runner.service`) under a dedicated
`gitea-runner` system user, which needs:
* Passwordless `sudo` for managing any `podman-compose@` instance —
`/etc/sudoers.d/gitea-runner-podman` grants `NOPASSWD` on
`systemctl {start,stop,restart} podman-compose@*.service` and
`podman rmi -f *` (root-owned systemd units, same idea as the
`docker` group membership gitea-runner already has for
`docker-compose@` services, just scoped explicitly since Podman has
no daemon/socket-group equivalent to grant blanket access through).
* Write access to `/srv/podman/maptool`.
Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single
label:
[source,yaml]
----
runner:
labels:
- "cube-host"
----
The `host` scheme means jobs run directly via the runner process's own
shell — no container spawned — which is exactly what a plain, single
purpose deploy runner needs.
== Puma's bind address
`config/puma.rb` binds to `127.0.0.1` by default (see
`docs/architecture.adoc`) — fine for local dev, but inside a container
that would make the app unreachable through the published port
(loopback is scoped to the container's own network namespace).
`docker-compose.yaml` sets `BIND_HOST="[::]"` to override that for the
deployed container specifically — the IPv6 wildcard, which (dual-stack
being the Linux default) also accepts IPv4 connections, covering both
with a single listener; the compose file's own port publishing
(`ports:`) is the network boundary there instead of Puma's bind
address.
== Secrets (`.env`)
`docker-compose.yaml` reads `SESSION_SECRET`, `GITEA_OAUTH_CLIENT_ID`,
`GITEA_OAUTH_CLIENT_SECRET` and `MAPTOOL_ALLOWED_GITEA_USER` from the
environment (see `docs/architecture.adoc` for what they're for) —
`podman compose` loads these automatically from a `.env` file next to
`docker-compose.yaml`, i.e. `/srv/podman/maptool/.env`. That file is
gitignored and dockerignored on purpose: nothing here ever puts
secrets in the repository or a built image layer.
It's not created by hand on the server — the deploy job's "Write
.env" step regenerates it on every deploy from repo-level Gitea
Actions secrets (*Settings → Actions → Secrets*): `SESSION_SECRET`,
`OAUTH_CLIENT_ID`, `OAUTH_CLIENT_SECRET`, `MAPTOOL_ALLOWED_GITEA_USER`.
The OAuth2 ones aren't named `GITEA_OAUTH_CLIENT_*` to match the `.env`
var names 1:1 — Gitea rejects secret names starting with `GITEA_` or
`GITHUB_` as reserved — but the workflow step maps them to the right
`.env` var names regardless. Gitea masks secret values in the job log
automatically. Set these once in the repo settings and every deploy
since keeps `.env` in sync — no server-side step needed beyond the
first-time OAuth2 application registration below.
`SESSION_SECRET` must stay the same value across deploys (regenerating
it every push would silently log everyone out every time `main`
changes) — generate it once (`openssl rand -hex 32`) and store that
same value as the secret from then on, don't regenerate per-deploy.
Register the OAuth2 application once, under any one Gitea account
(doesn't need to be an allowed user itself — any Gitea account can
authenticate through it, `MAPTOOL_ALLOWED_GITEA_USER` is what actually
decides who gets write access): *Settings → Applications → Manage
OAuth2 Applications → Create a new OAuth2 Application*, with redirect
URI `https://map.priesol.net/auth/callback`. That gives you the client
id/secret to store as the two `OAUTH_CLIENT_*` secrets above.
`MAPTOOL_ALLOWED_GITEA_USER` itself is a comma-separated list of Gitea
usernames, e.g. `randy,friend1,friend2` — not their emails, the
`login` field from Gitea's `/api/v1/user`.
== GNSS simulator
`docker-compose.yaml`'s `gnss-simulator` service (`bin/gnss_simulator.rb`,
also documented for local dev in `docs/development.adoc`) runs
alongside `maptool` in production too, sharing its network namespace
(`network_mode: "service:maptool"`) so it's reachable at the
hardcoded `127.0.0.1:2947` `Maptool::GpsdClient` expects — there's no
real GNSS receiver attached to the server, so without this the "GNSS"
control would just always show "no fix". It drives a loop around
Bratislava; remove the service block (nothing else needs to change)
whenever a real receiver/gpsd setup replaces it.
== Garmin LiveTrack session webhook
`POST /api/livetrack/session` (`session_id`, `session_token`, `secret` form
params) records a Garmin LiveTrack session to `data/livetrack.json`; `GET
/api/livetrack/sessions` lists them, newest first. Intended caller: a Google
Apps Script bound to the Gmail account that receives LiveTrack emails,
triggered on a timer, extracting the session id/token from the email and
posting them here — the endpoint itself doesn't know anything about email or
Garmin's API, just those two strings.
`GET /api/livetrack/sessions/:id/track` fetches and returns that session's
track as GeoJSON (a `LineString` plus a `Point` at the latest position) —
see `LiveTrackClient` (`lib/maptool/livetrack_client.rb`) for how, since
there's no usable REST call for this: Garmin's own API
(`livetrack.garmin.com/api/v2/...`) 403s a plain server-side HTTP request no
matter what headers it's given (Cloudflare bot protection, fingerprinting
at the TLS/HTTP level rather than anything header-based) — but the session
page itself isn't blocked, and Garmin's Next.js frontend renders the full
track server-side into it (as React Server Component data, not a follow-up
API call), so the client fetches that page and pulls the track out of the
embedded payload instead.
`secret` must match the `LIVETRACK_WEBHOOK_SECRET` repo secret (added to
`.env` the same way as the others above). Unlike the other secrets, this one
is optional — leaving it unset simply disables the endpoint (`halt 403` on
every call) rather than failing the deploy, since LiveTrack tracking isn't
core to the app.
The caller side is a Google Apps Script bound to the Gmail account that
receives the LiveTrack emails (script.google.com, "New project"), run on a
time-driven trigger:
[source,javascript]
----
const WEBHOOK_URL = "https://map.priesol.net/api/livetrack/session";
const WEBHOOK_SECRET = "PASTE_SAME_VALUE_AS_LIVETRACK_WEBHOOK_SECRET_IN_.env";
const PROCESSED_LABEL = "livetrack-processed";
function checkLiveTrackEmail() {
const label = GmailApp.getUserLabelByName(PROCESSED_LABEL) || GmailApp.createLabel(PROCESSED_LABEL);
const threads = GmailApp.search(`from:noreply@garmin.com subject:LiveTrack -label:${PROCESSED_LABEL}`);
threads.forEach(thread => {
// getBody() (raw HTML), not getPlainBody(): the email has no text/plain
// part, only text/html, and the link only exists as an href attribute
// — the auto-generated plain-text conversion isn't reliable for it.
const body = thread.getMessages()[0].getBody();
const match = body.match(/https:\/\/livetrack\.garmin\.com\/session\/([\w-]+)\/token\/([\w-]+)/);
if (match) {
UrlFetchApp.fetch(WEBHOOK_URL, {
method: "post",
payload: { session_id: match[1], session_token: match[2], secret: WEBHOOK_SECRET }
});
}
thread.addLabel(label); // stops it being reprocessed next run
});
}
----
The Gmail label (rather than marking the message read/unread) tracks which
threads are already handled without touching the user's own inbox state.
`subject:LiveTrack` matches regardless of the account's language — Garmin
keeps "LiveTrack" as-is in the subject even in a localized
("Sledujte LiveTrack používateľa ...") one.
Set the trigger from the editor: clock icon on the left (Triggers) → *Add
Trigger* → function `checkLiveTrackEmail`, time-driven, "Minutes timer",
every 5 minutes.
Verified against a real "started an activity" LiveTrack email
(2026-09-06): sender `noreply@garmin.com`, link
`https://livetrack.garmin.com/session/<uuid>/token/<token>`, and against
the resulting session page/track fetch (`LiveTrackClient`, same day).
Garmin also sends other LiveTrack notifications (e.g. activity ended) that
weren't checked — if one turns out to use a different link shape, the
webhook simply won't receive that session; nothing breaks.
== CI deploy job
`.gitea/workflows/ci.yml`'s `deploy` job runs after `check` passes, only
on a push to `main`, with `runs-on: cube-host`:
. Checks out the repo with plain `git` (`fetch` + `checkout`,
authenticated via a one-off `http.extraHeader`) rather than
`actions/checkout@v4` — a leftover from when this job ran in a
Node-less container; kept as-is since there's no reason to bring
Node back in for something this already does.
. Copies the checkout into `/srv/podman/maptool/` using plain `cp`/`rm`:
clears everything in the target except `data/`, copies the fresh
checkout over, then drops the copied `.git`.
. `sudo systemctl restart podman-compose@maptool.service`. The service
(`/etc/systemd/system/podman-compose@.service`, a systemd template
instantiated as `podman-compose@maptool.service`) owns the actual
container lifecycle — its `ExecStartPre` tears down the previous
containers/networks/volumes on every start, and its `ExecStart` runs
`podman compose up --build` (mirroring `docker-compose@.service`'s
own `--build`), so a plain restart always rebuilds from the code
just synced above while still reusing any layer whose inputs didn't
change (base image, `apt-get install`, `bundle install` when
`Gemfile`/`Gemfile.lock` are unchanged) as cache.
+
CAUTION: An earlier version of this step explicitly removed the
`maptool:latest` image (`podman rmi -f`) before restarting, to force a
rebuild against a service whose `ExecStart` didn't have `--build` yet.
That "worked" but also destroyed the build cache along with the image
— since nothing else referenced those layers, Podman fully deleted
them rather than leaving them as reusable cache the way Docker leaves
a retagged image's old layers behind — turning every deploy into a
full `apt-get`/`bundle install` from scratch (~3 minutes measured,
vs. ~20s before). Fixed by adding `--build` to the unit instead (see
above); don't reintroduce the `podman rmi` step.
. Polls `http://localhost:4567/api/health` for *this deploy's own*
revision (up to 5 minutes, the job fails if it never shows up)
before calling the deploy done — the restart above only confirms
`podman compose up --build` was launched (`Type=simple` has no
readiness protocol), not that the build actually succeeded and the
new code is what's answering; checking the revision specifically,
not just any 200, also catches a stuck old container that never got
torn down.