Truncate noisy build-failure output, fix the recurring terrain death
The web UI was showing a raw ~500k-character stdout/stderr dump on a failed build — go-pmtiles' extract logs one progress line per chunk, which is fine on a terminal (it overwrites itself) but not captured non-interactively. command_output now keeps just the tail, and the status line renders it with preserved line breaks in a scrollable box instead of one unreadable blob. Root cause of the actual failure (terrain build dying ~11 minutes in with an HTTP/2 stream reset): go-pmtiles' default --overfetch=0.05 merged this multi-country planet.pmtiles extract into a single ~60GB request with no way to resume after a dropped stream. --overfetch=0 splits it into many smaller requests instead.
This commit is contained in:
1 parent
237e5b2e54
commit
f18e5c19d3
5 files changed
+92
-13
No files matched your search
@@ -47,6 +47,21 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
File.write(@pmtiles_fail, "#!/bin/sh\necho 'boom' >&2\nexit 1\n")
|
||||
File.chmod(0o755, @pmtiles_fail)
|
||||
|
||||
# Mimics go-pmtiles' real extract failure mode: hundreds of progress
|
||||
# lines before the actual error — see command_output's tail-truncation.
|
||||
@pmtiles_fail_noisy = File.join(@bin_dir, "pmtiles-fail-noisy")
|
||||
File.write(@pmtiles_fail_noisy, <<~SH)
|
||||
#!/bin/sh
|
||||
i=0
|
||||
while [ "$i" -lt 200 ]; do
|
||||
echo "fetching $i dirs, $i chunks, 1 requests" >&2
|
||||
i=$((i + 1))
|
||||
done
|
||||
echo "stream error: stream ID 9; INTERNAL_ERROR" >&2
|
||||
exit 1
|
||||
SH
|
||||
File.chmod(0o755, @pmtiles_fail_noisy)
|
||||
|
||||
# osmium merge PBF1 PBF2 ... -o OUTPUT — output is always "-o"'s next
|
||||
# arg, unlike pmtiles' varying-position output above.
|
||||
@osmium_ok = File.join(@bin_dir, "osmium-ok")
|
||||
@@ -545,6 +560,22 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
assert_match(/exited with status/, status[:message])
|
||||
end
|
||||
|
||||
# The real trigger for this: a Mapterhorn extract that failed after ~7000
|
||||
# progress lines, producing a 500k+ character status[:message] that was
|
||||
# unreadable in the web UI.
|
||||
def test_build_terrain_truncates_a_noisy_command_failure
|
||||
select_regions("ES")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_fail_noisy
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread.join
|
||||
|
||||
status = Maptool::PmtilesBuilder.terrain_status
|
||||
assert_equal "error", status[:state]
|
||||
assert_match(/\(\d+ earlier lines omitted\)/, status[:message])
|
||||
assert_match(/stream error: stream ID 9/, status[:message])
|
||||
assert_operator status[:message].length, :<, 5000
|
||||
end
|
||||
|
||||
def test_build_terrain_raises_when_one_is_already_running
|
||||
select_regions("ES")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
|
||||
Reference in new issue
Block a user