Move deploy job to a dedicated native runner
The Docker-based runner's own container is too minimal for host-mode jobs (no rsync, docker CLI, or Node.js) and has no reliable path to the host filesystem or systemctl otherwise. Register a separate, native runner (label deploy:host) just for deployment, and point the deploy job at it.
This commit is contained in:
1 parent
71e638b459
commit
ae67cc72c7
2 files changed
+52
-100
No files matched your search
+12
-17
@@ -30,18 +30,18 @@ jobs:
|
|||||||
- name: JS syntax check
|
- name: JS syntax check
|
||||||
run: find public/js -name "*.js" -print0 | xargs -0 -n1 node --check
|
run: find public/js -name "*.js" -print0 | xargs -0 -n1 node --check
|
||||||
|
|
||||||
# Runs directly in the runner's own container (see the "host" label in the
|
# Handled by a separate, native (non-Docker) runner registered with the
|
||||||
# runner's config.yaml), not in an ephemeral per-job container — that's
|
# single "deploy:host" label (see docs/deployment.adoc) — it runs directly
|
||||||
# what gives this job access to /var/run/docker.sock (already mounted into
|
# on the server, so it has docker/git/etc. from the host, no socket/bind
|
||||||
# the runner container) and to /srv/docker (bind-mounted there too, see
|
# mount tricks needed. The Docker-based runner keeps handling `check`.
|
||||||
# docs/deployment.adoc). No systemd/sudo involved.
|
|
||||||
deploy:
|
deploy:
|
||||||
needs: check
|
needs: check
|
||||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||||
runs-on: host
|
runs-on: deploy
|
||||||
steps:
|
steps:
|
||||||
# Not actions/checkout@v4: it's a JS action and host-mode jobs run
|
# Not actions/checkout@v4: keeping this plain-git checkout from when
|
||||||
# directly in the runner container, which has git but no Node.js.
|
# the job ran in a Node-less container — no reason to add a Node
|
||||||
|
# dependency back in for something this already does.
|
||||||
- name: Checkout
|
- name: Checkout
|
||||||
run: |
|
run: |
|
||||||
rm -rf .git * .[!.]* 2>/dev/null
|
rm -rf .git * .[!.]* 2>/dev/null
|
||||||
@@ -51,15 +51,10 @@ jobs:
|
|||||||
fetch --depth 1 origin "${{ gitea.sha }}"
|
fetch --depth 1 origin "${{ gitea.sha }}"
|
||||||
git checkout -q FETCH_HEAD
|
git checkout -q FETCH_HEAD
|
||||||
|
|
||||||
# /srv/docker/maptool is the same path inside the runner container and
|
# Clears out everything in the target except data/ (uploaded GPX
|
||||||
# on the real host (1:1 bind mount) — required so the relative bind
|
# files, offline.pmtiles — never part of the git checkout in the
|
||||||
# mount `./data:/app/data` in docker-compose.yaml resolves to the
|
# first place, since it's gitignored) and copies the fresh checkout
|
||||||
# correct real host path when the daemon creates the container.
|
# over, dropping .git afterwards.
|
||||||
#
|
|
||||||
# No rsync in the runner image, so this clears out everything in the
|
|
||||||
# target except data/ (uploaded GPX files, offline.pmtiles — never
|
|
||||||
# part of the git checkout in the first place, since it's gitignored)
|
|
||||||
# and copies the fresh checkout over, dropping .git afterwards.
|
|
||||||
- name: Sync to /srv/docker/maptool
|
- name: Sync to /srv/docker/maptool
|
||||||
run: |
|
run: |
|
||||||
mkdir -p /srv/docker/maptool
|
mkdir -p /srv/docker/maptool
|
||||||
|
|||||||
+40
-83
@@ -3,10 +3,24 @@
|
|||||||
:toclevels: 2
|
:toclevels: 2
|
||||||
|
|
||||||
MapTool deploys as a Docker container on the same server the Gitea Actions
|
MapTool deploys as a Docker container on the same server the Gitea Actions
|
||||||
runner runs on. The `deploy` job in `.gitea/workflows/ci.yml` runs directly
|
runners run on. Two separate runners are registered against the Gitea
|
||||||
inside the runner's own container (see "The runner's host label" below) and
|
instance:
|
||||||
calls `docker compose` directly against the host's Docker daemon — no
|
|
||||||
systemd unit is involved for this app.
|
* The existing Docker-based runner (`gitea/runner:3`), labels
|
||||||
|
`ubuntu-latest`/`ubuntu-24.04`/`ubuntu-22.04` — runs the `check` job
|
||||||
|
in a fresh, isolated container per job, same as before.
|
||||||
|
* A dedicated native runner, installed directly on the host (no Docker
|
||||||
|
around it), label `deploy:host` — runs only the `deploy` job.
|
||||||
|
|
||||||
|
They're separate because the Docker-based runner's jobs (whether in
|
||||||
|
its usual isolated per-job containers, or tried directly inside its
|
||||||
|
own container via a `host`-scheme label) have no reliable access to
|
||||||
|
the host's filesystem, `systemctl`, or even a full toolset (its own
|
||||||
|
container image is missing `rsync`, `docker` CLI, Node.js — it's built
|
||||||
|
for spawning per-job containers via its `docker.sock` mount, not for
|
||||||
|
running arbitrary host commands). The native runner sidesteps all of
|
||||||
|
that: it *is* a host process, so `docker`, `git`, etc. are just
|
||||||
|
whatever's installed on the server.
|
||||||
|
|
||||||
== Layout on the server
|
== Layout on the server
|
||||||
|
|
||||||
@@ -19,82 +33,29 @@ systemd unit is involved for this app.
|
|||||||
`docker-compose.yaml`), so a redeploy (`docker compose up --build`)
|
`docker-compose.yaml`), so a redeploy (`docker compose up --build`)
|
||||||
never touches it.
|
never touches it.
|
||||||
|
|
||||||
== The runner's "host" label
|
== The native deploy runner
|
||||||
|
|
||||||
The Gitea Actions runner (`gitea/runner:3`) normally runs each job in a
|
Installed from the official binary (see
|
||||||
fresh, isolated container spawned via its mounted
|
https://docs.gitea.com/runner/installation/binary/), running as a
|
||||||
`/var/run/docker.sock` — that's how the `check` job runs (`runs-on:
|
systemd service (`gitea-runner.service`) under a dedicated
|
||||||
ubuntu-latest`). Such a job container has no access to the host
|
`gitea-runner` system user, which needs:
|
||||||
filesystem or to a host `systemctl`, so it can't reach
|
|
||||||
`/srv/docker/maptool` or manage a systemd unit.
|
|
||||||
|
|
||||||
The `deploy` job instead uses `runs-on: host`, a label configured in
|
* Docker access (member of the `docker` group).
|
||||||
the runner's own `config.yaml` that makes the runner (`gitea-runner`,
|
* Write access to `/srv/docker/maptool`.
|
||||||
the binary inside the `gitea/runner:3` image) execute the job's
|
|
||||||
steps directly inside the runner's *own* container, without spawning a
|
|
||||||
new one. That container already has `/var/run/docker.sock` mounted, so
|
|
||||||
`docker`/`docker compose` commands run there act directly on the
|
|
||||||
host's Docker daemon — this is the same runner used for `check`, just
|
|
||||||
with a second execution mode, not a separate runner.
|
|
||||||
|
|
||||||
For this to work, the runner's own `docker-compose.yaml` (the one that
|
Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single
|
||||||
starts the runner itself, not this app) needs one more bind mount so
|
label:
|
||||||
`/srv/docker` inside the runner container is the *same* path as on the
|
|
||||||
real host:
|
|
||||||
|
|
||||||
[source,yaml]
|
|
||||||
----
|
|
||||||
services:
|
|
||||||
runner:
|
|
||||||
image: docker.io/gitea/runner:3
|
|
||||||
# ...
|
|
||||||
volumes:
|
|
||||||
- ./config.yaml:/config.yaml
|
|
||||||
- ./data:/data
|
|
||||||
- /var/run/docker.sock:/var/run/docker.sock
|
|
||||||
- /srv/docker:/srv/docker
|
|
||||||
----
|
|
||||||
|
|
||||||
And the runner's `config.yaml` needs the `host` label added alongside
|
|
||||||
whatever docker-mode labels it already has. If `config.yaml` is empty
|
|
||||||
(no prior customization), generate a real default first rather than
|
|
||||||
hand-writing one — schemas vary across runner versions:
|
|
||||||
|
|
||||||
[source,shell]
|
|
||||||
----
|
|
||||||
docker exec <runner-container> gitea-runner config generate > config.yaml
|
|
||||||
----
|
|
||||||
|
|
||||||
(`config generate` — not `generate-config`, which is deprecated; and
|
|
||||||
the binary is `gitea-runner`, not `act_runner`.)
|
|
||||||
|
|
||||||
Then add the label to the generated `runner: labels:` list — the
|
|
||||||
generated defaults are commented out, and this list fully *replaces*
|
|
||||||
the runner's built-in defaults rather than adding to them, so all of
|
|
||||||
them need to be uncommented too, not just the new one:
|
|
||||||
|
|
||||||
[source,yaml]
|
[source,yaml]
|
||||||
----
|
----
|
||||||
runner:
|
runner:
|
||||||
labels:
|
labels:
|
||||||
- "host:host"
|
- "deploy:host"
|
||||||
- "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
|
|
||||||
- "ubuntu-24.04:docker://docker.gitea.com/runner-images:ubuntu-24.04"
|
|
||||||
- "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
|
|
||||||
----
|
----
|
||||||
|
|
||||||
The `1:1` path match matters: the deploy job's working directory
|
The `host` scheme means jobs run directly via the runner process's own
|
||||||
(`/srv/docker/maptool`) needs to resolve to the *same* path whether
|
shell — no container spawned — which is exactly what a plain, single
|
||||||
read from inside the runner container or by the Docker daemon on the
|
purpose deploy runner needs.
|
||||||
real host, because `docker-compose.yaml`'s relative bind mount
|
|
||||||
(`./data`) gets resolved to an absolute path by the `docker compose`
|
|
||||||
CLI *before* being sent to the daemon — if that path only existed
|
|
||||||
inside the runner container, the daemon (running on the bare host)
|
|
||||||
would create/mount the wrong (or a nonexistent) directory.
|
|
||||||
|
|
||||||
After editing both files, recreate the runner container so it picks
|
|
||||||
up the new volume and label (`docker compose up -d` in the runner's
|
|
||||||
own directory).
|
|
||||||
|
|
||||||
== Puma's bind address
|
== Puma's bind address
|
||||||
|
|
||||||
@@ -110,20 +71,16 @@ of Puma's bind address.
|
|||||||
== CI deploy job
|
== CI deploy job
|
||||||
|
|
||||||
`.gitea/workflows/ci.yml`'s `deploy` job runs after `check` passes, only
|
`.gitea/workflows/ci.yml`'s `deploy` job runs after `check` passes, only
|
||||||
on a push to `main`, with `runs-on: host`:
|
on a push to `main`, with `runs-on: deploy`:
|
||||||
|
|
||||||
. Checks out the repo with plain `git` (`fetch` + `checkout`,
|
. Checks out the repo with plain `git` (`fetch` + `checkout`,
|
||||||
authenticated via a one-off `http.extraHeader`), not
|
authenticated via a one-off `http.extraHeader`) rather than
|
||||||
`actions/checkout@v4` — that's a JS action, and host-mode jobs run
|
`actions/checkout@v4` — a leftover from when this job ran in a
|
||||||
directly in the runner container, which has `git` but no Node.js
|
Node-less container; kept as-is since there's no reason to bring
|
||||||
runtime to execute JS actions with.
|
Node back in for something this already does.
|
||||||
. Copies the checkout into `/srv/docker/maptool/` using plain
|
. Copies the checkout into `/srv/docker/maptool/` using plain `cp`/`rm`:
|
||||||
`cp`/`rm` (no `rsync` in the runner image): clears everything in the
|
clears everything in the target except `data/`, copies the fresh
|
||||||
target except `data/`, copies the fresh checkout over, then drops
|
checkout over, then drops the copied `.git`.
|
||||||
the copied `.git`.
|
|
||||||
. `cd /srv/docker/maptool && docker compose -p maptool up --build -d`
|
. `cd /srv/docker/maptool && docker compose -p maptool up --build -d`
|
||||||
— rebuilds the image and recreates the container if anything
|
— rebuilds the image and recreates the container if anything
|
||||||
changed; a no-op restart if nothing did.
|
changed; a no-op if nothing did.
|
||||||
|
|
||||||
No `sudo`, no systemd, no SSH — the runner container's existing access
|
|
||||||
to `/var/run/docker.sock` and (once added) `/srv/docker` is enough.
|
|
||||||
Reference in new issue
Block a user