Verify the deploy actually came up, not just that it was launched
systemctl restart (Type=simple, no readiness protocol) returns as soon as podman compose up --build forks, before the build or the container start have actually finished - a build failure or crash loop would still show as a green deploy job. Poll /api/health for up to 5 minutes (comfortably above the ~3.5min worst-case cold rebuild) and fail the job if it never answers.
This commit is contained in:
1 parent
a908cd7abc
commit
4982f04258
2 files changed
+24
No files matched your search
@@ -109,3 +109,22 @@ jobs:
|
|||||||
# (gitea-runner has passwordless sudo for this on cube-host).
|
# (gitea-runner has passwordless sudo for this on cube-host).
|
||||||
- name: Deploy
|
- name: Deploy
|
||||||
run: sudo systemctl restart podman-compose@maptool.service
|
run: sudo systemctl restart podman-compose@maptool.service
|
||||||
|
|
||||||
|
# systemctl restart above only confirms the process forked
|
||||||
|
# (Type=simple, no readiness protocol) — not that `podman compose up
|
||||||
|
# --build` actually finished and Puma is answering. Polls the health
|
||||||
|
# endpoint (cube-host itself, same machine the container's port is
|
||||||
|
# published on) instead of trusting that. 5 minutes covers even a
|
||||||
|
# from-scratch rebuild (Gemfile change, ~3.5 minutes measured) with
|
||||||
|
# room to spare; a cached rebuild is normally up within a minute.
|
||||||
|
- name: Wait for it to come up
|
||||||
|
run: |
|
||||||
|
for i in $(seq 1 30); do
|
||||||
|
if curl -sf http://localhost:4567/api/health >/dev/null; then
|
||||||
|
echo "Healthy after $(( (i - 1) * 10 ))s"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
sleep 10
|
||||||
|
done
|
||||||
|
echo "::error::maptool did not become healthy within 5 minutes"
|
||||||
|
exit 1
|
||||||
@@ -246,3 +246,8 @@ a retagged image's old layers behind — turning every deploy into a
|
|||||||
full `apt-get`/`bundle install` from scratch (~3 minutes measured,
|
full `apt-get`/`bundle install` from scratch (~3 minutes measured,
|
||||||
vs. ~20s before). Fixed by adding `--build` to the unit instead (see
|
vs. ~20s before). Fixed by adding `--build` to the unit instead (see
|
||||||
above); don't reintroduce the `podman rmi` step.
|
above); don't reintroduce the `podman rmi` step.
|
||||||
|
. Polls `http://localhost:4567/api/health` (up to 5 minutes, the job
|
||||||
|
fails if it never comes up) before calling the deploy done — the
|
||||||
|
restart above only confirms `podman compose up --build` was
|
||||||
|
launched (`Type=simple` has no readiness protocol), not that the
|
||||||
|
build actually succeeded and Puma is answering.
|
||||||
Reference in new issue
Block a user