Fix the real cause of slow podman deploys: add --build, drop podman rmi
podman-compose@.service was missing the --build flag that docker-compose@.service already has (added on cube.local: /etc/systemd/system/podman-compose@.service), so removing maptool:latest before every restart (my own earlier fix, meant to force a rebuild) also destroyed the build cache along with it - every deploy was redoing apt-get/bundle install from scratch (~3 min measured, vs ~20s before). --build lets a plain systemctl restart rebuild while still reusing unchanged layers, same as docker did, so the ci.yml deploy step collapses back to one line.
This commit is contained in:
1 parent
049d54e44e
commit
a908cd7abc
2 files changed
+37
-30
No files matched your search
+9
-15
@@ -99,19 +99,13 @@ jobs:
|
|||||||
|
|
||||||
# podman-compose@maptool.service (/etc/systemd/system/podman-compose@.service)
|
# podman-compose@maptool.service (/etc/systemd/system/podman-compose@.service)
|
||||||
# owns the container lifecycle — its ExecStartPre already tears down
|
# owns the container lifecycle — its ExecStartPre already tears down
|
||||||
# the previous containers/networks/volumes on every start, but not
|
# the previous containers/networks/volumes on every start, and its
|
||||||
# the built image, and its ExecStart runs a plain `podman compose up`
|
# ExecStart runs `podman compose up --build` (mirroring
|
||||||
# (deliberately no `--build`), which only builds when the image tag
|
# docker-compose@.service's own `--build`), so a plain restart always
|
||||||
# isn't already present locally. So a bare restart would just
|
# picks up the code just synced above while still reusing unchanged
|
||||||
# relaunch the previous image; removing that tag first forces a
|
# layers (apt-get/bundle install) as cache — unlike explicitly
|
||||||
# fresh build from the code just synced above — purely local, no
|
# removing the image first, which would delete that cache along with
|
||||||
# registry involved (same `maptool:latest` docker-compose.yaml
|
# it. sudo is needed since the unit is root's, not gitea-runner's own
|
||||||
# already builds/tags; podman rmi never talks to a registry for a
|
# (gitea-runner has passwordless sudo for this on cube-host).
|
||||||
# locally-built image). sudo is needed since the unit — and thus the
|
|
||||||
# image — is root's, not gitea-runner's own (gitea-runner has
|
|
||||||
# passwordless sudo for this on cube-host).
|
|
||||||
- name: Deploy
|
- name: Deploy
|
||||||
run: |
|
run: sudo systemctl restart podman-compose@maptool.service
|
||||||
sudo systemctl stop podman-compose@maptool.service
|
|
||||||
sudo podman rmi -f maptool:latest || true
|
|
||||||
sudo systemctl start podman-compose@maptool.service
|
|
||||||
+28
-15
@@ -46,10 +46,13 @@ https://docs.gitea.com/runner/installation/binary/), running as a
|
|||||||
systemd service (`gitea-runner.service`) under a dedicated
|
systemd service (`gitea-runner.service`) under a dedicated
|
||||||
`gitea-runner` system user, which needs:
|
`gitea-runner` system user, which needs:
|
||||||
|
|
||||||
* Passwordless `sudo` (confirmed via `sudo -l -U gitea-runner`) —
|
* Passwordless `sudo` for managing any `podman-compose@` instance —
|
||||||
`podman-compose@maptool.service` is a root-owned systemd unit (see
|
`/etc/sudoers.d/gitea-runner-podman` grants `NOPASSWD` on
|
||||||
below), and the image it builds lives in root's Podman storage, so
|
`systemctl {start,stop,restart} podman-compose@*.service` and
|
||||||
starting/stopping it and removing its image both require root.
|
`podman rmi -f *` (root-owned systemd units, same idea as the
|
||||||
|
`docker` group membership gitea-runner already has for
|
||||||
|
`docker-compose@` services, just scoped explicitly since Podman has
|
||||||
|
no daemon/socket-group equivalent to grant blanket access through).
|
||||||
* Write access to `/srv/podman/maptool`.
|
* Write access to `/srv/podman/maptool`.
|
||||||
|
|
||||||
Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single
|
Its `config.yaml` (`/etc/gitea-runner/config.yaml`) declares a single
|
||||||
@@ -222,14 +225,24 @@ on a push to `main`, with `runs-on: cube-host`:
|
|||||||
. Copies the checkout into `/srv/podman/maptool/` using plain `cp`/`rm`:
|
. Copies the checkout into `/srv/podman/maptool/` using plain `cp`/`rm`:
|
||||||
clears everything in the target except `data/`, copies the fresh
|
clears everything in the target except `data/`, copies the fresh
|
||||||
checkout over, then drops the copied `.git`.
|
checkout over, then drops the copied `.git`.
|
||||||
. Stops `podman-compose@maptool.service`, force-removes the
|
. `sudo systemctl restart podman-compose@maptool.service`. The service
|
||||||
`maptool:latest` image directly via `podman rmi`, then starts the
|
(`/etc/systemd/system/podman-compose@.service`, a systemd template
|
||||||
service again. The service (`/etc/systemd/system/podman-compose@.service`,
|
instantiated as `podman-compose@maptool.service`) owns the actual
|
||||||
a systemd template instantiated as `podman-compose@maptool.service`)
|
container lifecycle — its `ExecStartPre` tears down the previous
|
||||||
owns the actual container lifecycle — its `ExecStartPre` tears down
|
containers/networks/volumes on every start, and its `ExecStart` runs
|
||||||
the previous containers/networks/volumes on every start, and its
|
`podman compose up --build` (mirroring `docker-compose@.service`'s
|
||||||
`ExecStart` runs a plain `podman compose up` (deliberately no
|
own `--build`), so a plain restart always rebuilds from the code
|
||||||
`--build`), which only builds when the image tag isn't already
|
just synced above while still reusing any layer whose inputs didn't
|
||||||
present locally. Removing the tag first is what forces a fresh build
|
change (base image, `apt-get install`, `bundle install` when
|
||||||
from the code just synced above; without it, a restart alone would
|
`Gemfile`/`Gemfile.lock` are unchanged) as cache.
|
||||||
just relaunch the previous image unchanged.
|
+
|
||||||
|
CAUTION: An earlier version of this step explicitly removed the
|
||||||
|
`maptool:latest` image (`podman rmi -f`) before restarting, to force a
|
||||||
|
rebuild against a service whose `ExecStart` didn't have `--build` yet.
|
||||||
|
That "worked" but also destroyed the build cache along with the image
|
||||||
|
— since nothing else referenced those layers, Podman fully deleted
|
||||||
|
them rather than leaving them as reusable cache the way Docker leaves
|
||||||
|
a retagged image's old layers behind — turning every deploy into a
|
||||||
|
full `apt-get`/`bundle install` from scratch (~3 minutes measured,
|
||||||
|
vs. ~20s before). Fixed by adding `--build` to the unit instead (see
|
||||||
|
above); don't reintroduce the `podman rmi` step.
|
||||||
Reference in new issue
Block a user