Truncate noisy build-failure output, fix the recurring terrain death
CI / check (push) Failing after 4s
CI / deploy (push) Skipped

The web UI was showing a raw ~500k-character stdout/stderr dump on a
failed build — go-pmtiles' extract logs one progress line per chunk,
which is fine on a terminal (it overwrites itself) but not captured
non-interactively. command_output now keeps just the tail, and the
status line renders it with preserved line breaks in a scrollable box
instead of one unreadable blob.

Root cause of the actual failure (terrain build dying ~11 minutes in
with an HTTP/2 stream reset): go-pmtiles' default --overfetch=0.05
merged this multi-country planet.pmtiles extract into a single ~60GB
request with no way to resume after a dropped stream. --overfetch=0
splits it into many smaller requests instead.
This commit is contained in:
randy committed 2026-09-12 19:41:45 +02:00
1 parent 237e5b2e54
commit f18e5c19d3
5 files changed
+92 -13

No files matched your search

+31
View File
@@ -47,6 +47,21 @@ class PmtilesBuilderTest < Minitest::Test
File.write(@pmtiles_fail, "#!/bin/sh\necho 'boom' >&2\nexit 1\n")
File.chmod(0o755, @pmtiles_fail)
# Mimics go-pmtiles' real extract failure mode: hundreds of progress
# lines before the actual error — see command_output's tail-truncation.
@pmtiles_fail_noisy = File.join(@bin_dir, "pmtiles-fail-noisy")
File.write(@pmtiles_fail_noisy, <<~SH)
#!/bin/sh
i=0
while [ "$i" -lt 200 ]; do
echo "fetching $i dirs, $i chunks, 1 requests" >&2
i=$((i + 1))
done
echo "stream error: stream ID 9; INTERNAL_ERROR" >&2
exit 1
SH
File.chmod(0o755, @pmtiles_fail_noisy)
# osmium merge PBF1 PBF2 ... -o OUTPUT — output is always "-o"'s next
# arg, unlike pmtiles' varying-position output above.
@osmium_ok = File.join(@bin_dir, "osmium-ok")
@@ -545,6 +560,22 @@ class PmtilesBuilderTest < Minitest::Test
assert_match(/exited with status/, status[:message])
end
# The real trigger for this: a Mapterhorn extract that failed after ~7000
# progress lines, producing a 500k+ character status[:message] that was
# unreadable in the web UI.
def test_build_terrain_truncates_a_noisy_command_failure
select_regions("ES")
ENV["PMTILES_BIN"] = @pmtiles_fail_noisy
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
thread.join
status = Maptool::PmtilesBuilder.terrain_status
assert_equal "error", status[:state]
assert_match(/\(\d+ earlier lines omitted\)/, status[:message])
assert_match(/stream error: stream ID 9/, status[:message])
assert_operator status[:message].length, :<, 5000
end
def test_build_terrain_raises_when_one_is_already_running
select_regions("ES")
ENV["PMTILES_BIN"] = @pmtiles_ok