Fix the real cause of slow podman deploys: add --build, drop podman rmi
CI / check (push) Successful in 44s
CI / deploy (push) Successful in 3s

podman-compose@.service was missing the --build flag that
docker-compose@.service already has (added on cube.local:
/etc/systemd/system/podman-compose@.service), so removing
maptool:latest before every restart (my own earlier fix, meant to
force a rebuild) also destroyed the build cache along with it - every
deploy was redoing apt-get/bundle install from scratch (~3 min
measured, vs ~20s before). --build lets a plain systemctl restart
rebuild while still reusing unchanged layers, same as docker did, so
the ci.yml deploy step collapses back to one line.
This commit is contained in:
randy committed 2026-09-08 21:12:54 +02:00
1 parent 049d54e44e
commit a908cd7abc
2 files changed
+37 -30

No files matched your search

+9 -15
View File
@@ -99,19 +99,13 @@ jobs:
# podman-compose@maptool.service (/etc/systemd/system/podman-compose@.service) # podman-compose@maptool.service (/etc/systemd/system/podman-compose@.service)
# owns the container lifecycle — its ExecStartPre already tears down # owns the container lifecycle — its ExecStartPre already tears down
# the previous containers/networks/volumes on every start, but not # the previous containers/networks/volumes on every start, and its
# the built image, and its ExecStart runs a plain `podman compose up` # ExecStart runs `podman compose up --build` (mirroring
# (deliberately no `--build`), which only builds when the image tag # docker-compose@.service's own `--build`), so a plain restart always
# isn't already present locally. So a bare restart would just # picks up the code just synced above while still reusing unchanged
# relaunch the previous image; removing that tag first forces a # layers (apt-get/bundle install) as cache — unlike explicitly
# fresh build from the code just synced above — purely local, no # removing the image first, which would delete that cache along with
# registry involved (same `maptool:latest` docker-compose.yaml # it. sudo is needed since the unit is root's, not gitea-runner's own
# already builds/tags; podman rmi never talks to a registry for a # (gitea-runner has passwordless sudo for this on cube-host).
# locally-built image). sudo is needed since the unit — and thus the
# image — is root's, not gitea-runner's own (gitea-runner has
# passwordless sudo for this on cube-host).
- name: Deploy - name: Deploy
run: | run: sudo systemctl restart podman-compose@maptool.service
sudo systemctl stop podman-compose@maptool.service
sudo podman rmi -f maptool:latest || true
sudo systemctl start podman-compose@maptool.service
+28 -15
View File
@@ -46,10 +46,13 @@ https://docs.gitea.com/runner/installation/binary/), running as a
systemd service (`gitea-runner.service`) under a dedicated systemd service (`gitea-runner.service`) under a dedicated
`gitea-runner` system user, which needs: `gitea-runner` system user, which needs:
* Passwordless `sudo` (confirmed via `sudo -l -U gitea-runner`) — * Passwordless `sudo` for managing any `podman-compose@` instance —
`podman-compose@maptool.service` is a root-owned systemd unit (see `/etc/sudoers.d/gitea-runner-podman` grants `NOPASSWD` on
below), and the image it builds lives in root's Podman storage, so `systemctl {start,stop,restart} podman-compose@*.service` and
starting/stopping it and removing its image both require root. `podman rmi -f *` (root-owned systemd units, same idea as the
`docker` group membership gitea-runner already has for
`docker-compose@` services, just scoped explicitly since Podman has
no daemon/socket-group equivalent to grant blanket access through).
* Write access to `/srv/podman/maptool`. * Write access to `/srv/podman/maptool`.
Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single
@@ -222,14 +225,24 @@ on a push to `main`, with `runs-on: cube-host`:
. Copies the checkout into `/srv/podman/maptool/` using plain `cp`/`rm`: . Copies the checkout into `/srv/podman/maptool/` using plain `cp`/`rm`:
clears everything in the target except `data/`, copies the fresh clears everything in the target except `data/`, copies the fresh
checkout over, then drops the copied `.git`. checkout over, then drops the copied `.git`.
. Stops `podman-compose@maptool.service`, force-removes the . `sudo systemctl restart podman-compose@maptool.service`. The service
`maptool:latest` image directly via `podman rmi`, then starts the (`/etc/systemd/system/podman-compose@.service`, a systemd template
service again. The service (`/etc/systemd/system/podman-compose@.service`, instantiated as `podman-compose@maptool.service`) owns the actual
a systemd template instantiated as `podman-compose@maptool.service`) container lifecycle — its `ExecStartPre` tears down the previous
owns the actual container lifecycle — its `ExecStartPre` tears down containers/networks/volumes on every start, and its `ExecStart` runs
the previous containers/networks/volumes on every start, and its `podman compose up --build` (mirroring `docker-compose@.service`'s
`ExecStart` runs a plain `podman compose up` (deliberately no own `--build`), so a plain restart always rebuilds from the code
`--build`), which only builds when the image tag isn't already just synced above while still reusing any layer whose inputs didn't
present locally. Removing the tag first is what forces a fresh build change (base image, `apt-get install`, `bundle install` when
from the code just synced above; without it, a restart alone would `Gemfile`/`Gemfile.lock` are unchanged) as cache.
just relaunch the previous image unchanged. +
CAUTION: An earlier version of this step explicitly removed the
`maptool:latest` image (`podman rmi -f`) before restarting, to force a
rebuild against a service whose `ExecStart` didn't have `--build` yet.
That "worked" but also destroyed the build cache along with the image
— since nothing else referenced those layers, Podman fully deleted
them rather than leaving them as reusable cache the way Docker leaves
a retagged image's old layers behind — turning every deploy into a
full `apt-get`/`bundle install` from scratch (~3 minutes measured,
vs. ~20s before). Fixed by adding `--build` to the unit instead (see
above); don't reintroduce the `podman rmi` step.