Renovate no longer needs Docker access for a one-shot docker-run job: switched to setup-node + npm install -g renovate (matching labelprint's workflow), so it fits the same brick-build runner as check instead of the stale "deploy" label. Also note down the build-log readability/API gaps flagged from a real failed terrain build.
289 lines
14 KiB
Plaintext
289 lines
14 KiB
Plaintext
= Deployment
|
|
:toc:
|
|
:toclevels: 2
|
|
|
|
MapTool deploys as a Podman container (via `podman compose`, driven by a
|
|
`podman-compose@.service` systemd template — see below) on the same server
|
|
the Gitea Actions runners run on. Two separate runners are registered
|
|
against the Gitea instance:
|
|
|
|
* The existing Docker-based runner (`gitea/runner:3`), label
|
|
`brick-build` — runs the `check` and `renovate` jobs in a fresh,
|
|
isolated container per job, same as before.
|
|
* A dedicated native runner, installed directly on the host (no Docker
|
|
around it), label `brick-host` — runs only the `deploy` job.
|
|
|
|
They're separate because the Docker-based runner's jobs (whether in
|
|
its usual isolated per-job containers, or tried directly inside its
|
|
own container via a `host`-scheme label) have no reliable access to
|
|
the host's filesystem, `systemctl`, or even a full toolset (its own
|
|
container image is missing `rsync`, `docker` CLI, Node.js — it's built
|
|
for spawning per-job containers via its `docker.sock` mount, not for
|
|
running arbitrary host commands). The native runner sidesteps all of
|
|
that: it *is* a host process, so `docker`, `git`, etc. are just
|
|
whatever's installed on the server.
|
|
|
|
== Layout on the server
|
|
|
|
`/srv/podman/maptool/` (`/srv/podman` is a symlink to `/srv/docker`, a
|
|
leftover from before the Docker → Podman migration — kept around so
|
|
existing paths/tooling that still assume `/srv/docker` keep working;
|
|
expect this to flip the other way, `/srv/docker` symlinked to
|
|
`/srv/podman`, once nothing depends on the old name anymore) holds:
|
|
|
|
* `build/` — a checkout of this repository (kept in sync by the
|
|
deploy job, see below), wiped and fully replaced every deploy —
|
|
`Dockerfile`, `docker-compose.yaml`, application code.
|
|
* `docker-compose.yaml` — a relative symlink to `build/docker-compose.yaml`,
|
|
purely so it's where a human poking around expects to find it, at
|
|
the top rather than a level down. Also what makes the layout work at
|
|
all: `podman compose`/`docker compose` resolve a relative volume
|
|
path (`./data` below) against the *symlink's own* directory, not
|
|
where it points — confirmed against plain `docker-compose config`,
|
|
not assumed — so `./data:/app/data` in the compose file keeps
|
|
meaning `/srv/podman/maptool/data`, and
|
|
`podman-compose@.service`'s `WorkingDirectory=/srv/podman/maptool`
|
|
needs no change either.
|
|
* `data/`, *not* part of the checkout sync (nothing in `build/`
|
|
references it, and the sync step doesn't need to special-case it
|
|
the way an earlier version of this layout did) — uploaded GPX
|
|
files, `offline.pmtiles`, `terrain.pmtiles`, and
|
|
`kiwix/wikipedia.zim` (see `docs/development.adoc` for each).
|
|
Persisted via that bind mount, so a redeploy never touches it.
|
|
Also where a private import plugin belongs (`data/plugins/<name>/`,
|
|
see `lib/maptool/plugins.rb`'s `PLUGINS_DIRS`) — `build/plugins/`
|
|
only holds what's checked into this public repo, and gets wiped and
|
|
replaced by the checkout sync on every deploy same as the rest of
|
|
`build/`.
|
|
`data/` (and the kiwix ZIM path) carries a `:Z` SELinux relabel in
|
|
`docker-compose.yaml`'s bind mounts — without it, Podman on an
|
|
SELinux-enforcing host denies the container write access to the
|
|
bind-mounted directory regardless of its Unix permissions (surfaces
|
|
as a plain `Permission denied` in the app's own logs, nothing
|
|
SELinux-labeled unless you go looking in `audit.log`); harmless/
|
|
ignored on a host where SELinux isn't enforcing.
|
|
* `.env` — regenerated every deploy (see below), lives here rather
|
|
than in `build/` for the same reason as the symlink: `docker
|
|
compose`/`podman compose` auto-load `.env` from the compose file's
|
|
own directory (as invoked, i.e. following the symlink's location,
|
|
same story as the volume path above).
|
|
|
|
== The native deploy runner
|
|
|
|
Installed from the official binary (see
|
|
https://docs.gitea.com/runner/installation/binary/), running as a
|
|
systemd service (`gitea-runner.service`) under a dedicated
|
|
`gitea-runner` system user, which needs:
|
|
|
|
* Passwordless `sudo` for managing any `podman-compose@` instance —
|
|
`/etc/sudoers.d/gitea-runner-podman` grants `NOPASSWD` on
|
|
`systemctl {start,stop,restart} podman-compose@*.service` and
|
|
`podman rmi -f *` (root-owned systemd units, same idea as the
|
|
`docker` group membership gitea-runner already has for
|
|
`docker-compose@` services, just scoped explicitly since Podman has
|
|
no daemon/socket-group equivalent to grant blanket access through).
|
|
* Write access to `/srv/podman/maptool`.
|
|
|
|
Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single
|
|
label:
|
|
|
|
[source,yaml]
|
|
----
|
|
runner:
|
|
labels:
|
|
- "brick-host"
|
|
----
|
|
|
|
The `host` scheme means jobs run directly via the runner process's own
|
|
shell — no container spawned — which is exactly what a plain, single
|
|
purpose deploy runner needs.
|
|
|
|
== Puma's bind address
|
|
|
|
`config/puma.rb` binds to `127.0.0.1` by default (see
|
|
`docs/architecture.adoc`) — fine for local dev, but inside a container
|
|
that would make the app unreachable through the published port
|
|
(loopback is scoped to the container's own network namespace).
|
|
`docker-compose.yaml` sets `BIND_HOST="[::]"` to override that for the
|
|
deployed container specifically — the IPv6 wildcard, which (dual-stack
|
|
being the Linux default) also accepts IPv4 connections, covering both
|
|
with a single listener; the compose file's own port publishing
|
|
(`ports:`) is the network boundary there instead of Puma's bind
|
|
address.
|
|
|
|
== Secrets (`.env`)
|
|
|
|
`docker-compose.yaml` reads `SESSION_SECRET`, `GITEA_OAUTH_CLIENT_ID`,
|
|
`GITEA_OAUTH_CLIENT_SECRET` and `MAPTOOL_ALLOWED_GITEA_USER` from the
|
|
environment (see `docs/architecture.adoc` for what they're for) —
|
|
`podman compose` loads these automatically from a `.env` file next to
|
|
`docker-compose.yaml`, i.e. `/srv/podman/maptool/.env`. That file is
|
|
gitignored and dockerignored on purpose: nothing here ever puts
|
|
secrets in the repository or a built image layer.
|
|
|
|
It's not created by hand on the server — the deploy job's "Write
|
|
.env" step regenerates it on every deploy from repo-level Gitea
|
|
Actions secrets (*Settings → Actions → Secrets*): `SESSION_SECRET`,
|
|
`OAUTH_CLIENT_ID`, `OAUTH_CLIENT_SECRET`, `MAPTOOL_ALLOWED_GITEA_USER`.
|
|
The OAuth2 ones aren't named `GITEA_OAUTH_CLIENT_*` to match the `.env`
|
|
var names 1:1 — Gitea rejects secret names starting with `GITEA_` or
|
|
`GITHUB_` as reserved — but the workflow step maps them to the right
|
|
`.env` var names regardless. Gitea masks secret values in the job log
|
|
automatically. Set these once in the repo settings and every deploy
|
|
since keeps `.env` in sync — no server-side step needed beyond the
|
|
first-time OAuth2 application registration below.
|
|
|
|
`SESSION_SECRET` must stay the same value across deploys (regenerating
|
|
it every push would silently log everyone out every time `main`
|
|
changes) — generate it once (`openssl rand -hex 32`) and store that
|
|
same value as the secret from then on, don't regenerate per-deploy.
|
|
|
|
Register the OAuth2 application once, under any one Gitea account
|
|
(doesn't need to be an allowed user itself — any Gitea account can
|
|
authenticate through it, `MAPTOOL_ALLOWED_GITEA_USER` is what actually
|
|
decides who gets write access): *Settings → Applications → Manage
|
|
OAuth2 Applications → Create a new OAuth2 Application*, with redirect
|
|
URI `https://map.priesol.net/auth/callback`. That gives you the client
|
|
id/secret to store as the two `OAUTH_CLIENT_*` secrets above.
|
|
|
|
`MAPTOOL_ALLOWED_GITEA_USER` itself is a comma-separated list of Gitea
|
|
usernames, e.g. `randy,friend1,friend2` — not their emails, the
|
|
`login` field from Gitea's `/api/v1/user`.
|
|
|
|
== GNSS simulator
|
|
|
|
`docker-compose.yaml`'s `gnss-simulator` service (`bin/gnss_simulator.rb`,
|
|
also documented for local dev in `docs/development.adoc`) runs
|
|
alongside `maptool` in production too, sharing its network namespace
|
|
(`network_mode: "service:maptool"`) so it's reachable at the
|
|
hardcoded `127.0.0.1:2947` `Maptool::GpsdClient` expects — there's no
|
|
real GNSS receiver attached to the server, so without this the "GNSS"
|
|
control would just always show "no fix". It drives a loop around
|
|
Bratislava; remove the service block (nothing else needs to change)
|
|
whenever a real receiver/gpsd setup replaces it.
|
|
|
|
== Garmin LiveTrack session webhook
|
|
|
|
`POST /api/livetrack/session` (`session_id`, `session_token`, `secret` form
|
|
params) records a Garmin LiveTrack session to `data/livetrack.json`; `GET
|
|
/api/livetrack/sessions` lists them, newest first. Intended caller: a Google
|
|
Apps Script bound to the Gmail account that receives LiveTrack emails,
|
|
triggered on a timer, extracting the session id/token from the email and
|
|
posting them here — the endpoint itself doesn't know anything about email or
|
|
Garmin's API, just those two strings.
|
|
|
|
`GET /api/livetrack/sessions/:id/track` fetches and returns that session's
|
|
track as GeoJSON (a `LineString` plus a `Point` at the latest position) —
|
|
see `LiveTrackClient` (`lib/maptool/livetrack_client.rb`) for how, since
|
|
there's no usable REST call for this: Garmin's own API
|
|
(`livetrack.garmin.com/api/v2/...`) 403s a plain server-side HTTP request no
|
|
matter what headers it's given (Cloudflare bot protection, fingerprinting
|
|
at the TLS/HTTP level rather than anything header-based) — but the session
|
|
page itself isn't blocked, and Garmin's Next.js frontend renders the full
|
|
track server-side into it (as React Server Component data, not a follow-up
|
|
API call), so the client fetches that page and pulls the track out of the
|
|
embedded payload instead.
|
|
|
|
`secret` must match the `LIVETRACK_WEBHOOK_SECRET` repo secret (added to
|
|
`.env` the same way as the others above). Unlike the other secrets, this one
|
|
is optional — leaving it unset simply disables the endpoint (`halt 403` on
|
|
every call) rather than failing the deploy, since LiveTrack tracking isn't
|
|
core to the app.
|
|
|
|
The caller side is a Google Apps Script bound to the Gmail account that
|
|
receives the LiveTrack emails (script.google.com, "New project"), run on a
|
|
time-driven trigger:
|
|
|
|
[source,javascript]
|
|
----
|
|
const WEBHOOK_URL = "https://map.priesol.net/api/livetrack/session";
|
|
const WEBHOOK_SECRET = "PASTE_SAME_VALUE_AS_LIVETRACK_WEBHOOK_SECRET_IN_.env";
|
|
const PROCESSED_LABEL = "livetrack-processed";
|
|
|
|
function checkLiveTrackEmail() {
|
|
const label = GmailApp.getUserLabelByName(PROCESSED_LABEL) || GmailApp.createLabel(PROCESSED_LABEL);
|
|
const threads = GmailApp.search(`from:noreply@garmin.com subject:LiveTrack -label:${PROCESSED_LABEL}`);
|
|
|
|
threads.forEach(thread => {
|
|
// getBody() (raw HTML), not getPlainBody(): the email has no text/plain
|
|
// part, only text/html, and the link only exists as an href attribute
|
|
// — the auto-generated plain-text conversion isn't reliable for it.
|
|
const body = thread.getMessages()[0].getBody();
|
|
const match = body.match(/https:\/\/livetrack\.garmin\.com\/session\/([\w-]+)\/token\/([\w-]+)/);
|
|
|
|
if (match) {
|
|
UrlFetchApp.fetch(WEBHOOK_URL, {
|
|
method: "post",
|
|
payload: { session_id: match[1], session_token: match[2], secret: WEBHOOK_SECRET }
|
|
});
|
|
}
|
|
|
|
thread.addLabel(label); // stops it being reprocessed next run
|
|
});
|
|
}
|
|
----
|
|
|
|
The Gmail label (rather than marking the message read/unread) tracks which
|
|
threads are already handled without touching the user's own inbox state.
|
|
`subject:LiveTrack` matches regardless of the account's language — Garmin
|
|
keeps "LiveTrack" as-is in the subject even in a localized
|
|
("Sledujte LiveTrack používateľa ...") one.
|
|
|
|
Set the trigger from the editor: clock icon on the left (Triggers) → *Add
|
|
Trigger* → function `checkLiveTrackEmail`, time-driven, "Minutes timer",
|
|
every 5 minutes.
|
|
|
|
Verified against a real "started an activity" LiveTrack email
|
|
(2026-09-06): sender `noreply@garmin.com`, link
|
|
`https://livetrack.garmin.com/session/<uuid>/token/<token>`, and against
|
|
the resulting session page/track fetch (`LiveTrackClient`, same day).
|
|
Garmin also sends other LiveTrack notifications (e.g. activity ended) that
|
|
weren't checked — if one turns out to use a different link shape, the
|
|
webhook simply won't receive that session; nothing breaks.
|
|
|
|
== CI deploy job
|
|
|
|
`.gitea/workflows/ci.yml`'s `deploy` job runs after `check` passes, only
|
|
on a push to `main`, with `runs-on: brick-host`:
|
|
|
|
. Checks out the repo with plain `git` (`fetch` + `checkout`,
|
|
authenticated via a one-off `http.extraHeader`) rather than
|
|
`actions/checkout@v4` — a leftover from when this job ran in a
|
|
Node-less container; kept as-is since there's no reason to bring
|
|
Node back in for something this already does.
|
|
. Wipes and replaces `/srv/podman/maptool/build/` with the fresh
|
|
checkout (plain `cp`/`rm`, dropping the copied `.git`), then
|
|
`ln -sf`s `docker-compose.yaml` up to it — self-healing (harmless
|
|
to redo every deploy) and, being a fresh checkout each time, nothing
|
|
in `build/` needs preserving the way `data/`/`.env` (both a level
|
|
up, see "Layout on the server" above) do.
|
|
. `sudo systemctl restart podman-compose@maptool.service`. The service
|
|
(`/etc/systemd/system/podman-compose@.service`, a systemd template
|
|
instantiated as `podman-compose@maptool.service`) owns the actual
|
|
container lifecycle — its `ExecStartPre` tears down the previous
|
|
containers/networks/volumes on every start, and its `ExecStart` runs
|
|
`podman compose up --build` (mirroring `docker-compose@.service`'s
|
|
own `--build`), so a plain restart always rebuilds from the code
|
|
just synced above while still reusing any layer whose inputs didn't
|
|
change (base image, `apt-get install`, `bundle install` when
|
|
`Gemfile`/`Gemfile.lock` are unchanged) as cache.
|
|
+
|
|
CAUTION: An earlier version of this step explicitly removed the
|
|
`maptool:latest` image (`podman rmi -f`) before restarting, to force a
|
|
rebuild against a service whose `ExecStart` didn't have `--build` yet.
|
|
That "worked" but also destroyed the build cache along with the image
|
|
— since nothing else referenced those layers, Podman fully deleted
|
|
them rather than leaving them as reusable cache the way Docker leaves
|
|
a retagged image's old layers behind — turning every deploy into a
|
|
full `apt-get`/`bundle install` from scratch (~3 minutes measured,
|
|
vs. ~20s before). Fixed by adding `--build` to the unit instead (see
|
|
above); don't reintroduce the `podman rmi` step.
|
|
. Polls `http://localhost:4567/api/health` for *this deploy's own*
|
|
revision (up to 5 minutes, the job fails if it never shows up)
|
|
before calling the deploy done — the restart above only confirms
|
|
`podman compose up --build` was launched (`Type=simple` has no
|
|
readiness protocol), not that the build actually succeeded and the
|
|
new code is what's answering; checking the revision specifically,
|
|
not just any 200, also catches a stuck old container that never got
|
|
torn down.
|