Add a country-based region picker for offline data
Replaces "regenerate for the current viewport" with what Locus/OsmAnd- style apps give you: tick which countries you want offline (a Natural Earth-derived checklist, public domain, vendored), Apply rebuilds offline.pmtiles/terrain.pmtiles to cover the whole selection at once. Multiple regions can be offline simultaneously (e.g. Slovakia + Austria for a trip that crosses the border) — the selection itself is the source of truth (data/offline_regions.json), not a set of per-country files, so "remove a country" and "refresh a country" are both just "change the selection, then Apply." Confirmed against go-pmtiles' own source (not just its docs) before committing to this shape: `extract --region=<geojson>` treats a FeatureCollection's features as their union already (no dissolve/union math needed on our side — pmtiles/region.go's UnmarshalRegion just concatenates every feature's rings), and `merge` hard-errors on any overlapping tile id between inputs (merge.go's prepareInputs). That second fact rules out "extract each country separately, merge them all" — adjacent countries' region-extracts would share border tiles, since extraction copies whole tiles rather than clipping vector content to the polygon. The actual fix: extract per *source archive* (the Protomaps build, each intersecting Mapterhorn chunk), each call covering the whole current selection via one --region= file — the terrain pipeline already extracted-then-merged per source this way, it just needed a multi-country region instead of a single bbox.
This commit is contained in:
1 parent
f89dea5805
commit
53c9a82af2
10 files changed
+441
-169
No files matched your search
@@ -223,18 +223,38 @@ module Maptool
|
||||
send_file OFFLINE_PMTILES_PATH, type: "application/octet-stream"
|
||||
end
|
||||
|
||||
# Regenerates offline.pmtiles for the given bbox (the current map
|
||||
# viewport, same "visible area is the area" convention as the Overpass
|
||||
# POI plugin) by extracting it from Protomaps' own daily full-planet
|
||||
# basemap build — see lib/maptool/pmtiles_builder.rb. Runs in the
|
||||
# background; poll build-status below for progress.
|
||||
post "/api/offline-tiles/build" do
|
||||
# Lists the selectable countries and which of them are currently
|
||||
# selected (data/offline_regions.json) — read-only, no login needed,
|
||||
# same tier as the plain availability checks above.
|
||||
get "/api/offline-regions" do
|
||||
json available: PmtilesBuilder.available_regions, selected: PmtilesBuilder.selected_regions(DATA_DIR)
|
||||
end
|
||||
|
||||
# Just updates the selection — does not itself start a build, so
|
||||
# ticking several countries doesn't fire off several rebuilds. The
|
||||
# frontend follows this with the two build routes below once the user
|
||||
# is done choosing.
|
||||
post "/api/offline-regions" do
|
||||
require_login!
|
||||
bbox = %w[south west north east].map { |key| params[key].to_s }
|
||||
halt 400, json(error: "south, west, north, east are required") if bbox.any?(&:empty?)
|
||||
codes = Array(params[:codes])
|
||||
|
||||
begin
|
||||
PmtilesBuilder.build_offline(bbox.map(&:to_f), data_dir: DATA_DIR)
|
||||
PmtilesBuilder.set_selected_regions(DATA_DIR, codes)
|
||||
rescue PmtilesBuilder::BuildError => e
|
||||
halt 400, json(error: e.message)
|
||||
end
|
||||
json ok: true
|
||||
end
|
||||
|
||||
# Regenerates offline.pmtiles for the currently selected regions (see
|
||||
# POST /api/offline-regions above) by extracting them from Protomaps'
|
||||
# own daily full-planet basemap build — see
|
||||
# lib/maptool/pmtiles_builder.rb. Runs in the background; poll
|
||||
# build-status below for progress.
|
||||
post "/api/offline-tiles/build" do
|
||||
require_login!
|
||||
begin
|
||||
PmtilesBuilder.build_offline(data_dir: DATA_DIR)
|
||||
rescue PmtilesBuilder::BuildError => e
|
||||
halt 409, json(error: e.message)
|
||||
end
|
||||
@@ -260,11 +280,8 @@ module Maptool
|
||||
|
||||
post "/api/terrain-tiles/build" do
|
||||
require_login!
|
||||
bbox = %w[south west north east].map { |key| params[key].to_s }
|
||||
halt 400, json(error: "south, west, north, east are required") if bbox.any?(&:empty?)
|
||||
|
||||
begin
|
||||
PmtilesBuilder.build_terrain(bbox.map(&:to_f), data_dir: DATA_DIR)
|
||||
PmtilesBuilder.build_terrain(data_dir: DATA_DIR)
|
||||
rescue PmtilesBuilder::BuildError => e
|
||||
halt 409, json(error: e.message)
|
||||
end
|
||||
|
||||
+37
-15
@@ -85,25 +85,47 @@ size rather than assuming one, so this isn't a problem either way.
|
||||
|
||||
A logged-in user can also generate both files above from the app
|
||||
itself, no local tooling needed — the sidebar's "Offline data" icon
|
||||
(Offline hub group) shows each file's status (size, last updated) and
|
||||
a "Regenerate for current view" button. The region is always the
|
||||
current map viewport, same "no area-picker, the visible area is the
|
||||
area" convention the Overpass POI plugin already uses below. This
|
||||
wraps exactly the manual `go-pmtiles` workflows documented above
|
||||
(Offline hub group) shows a checklist of European countries
|
||||
(`lib/maptool/data/europe_countries.geojson`, Natural Earth 110m
|
||||
admin-0, public domain) plus each file's status (size, last updated).
|
||||
Ticking countries and clicking "Apply changes" saves the selection
|
||||
(`data/offline_regions.json`, an array of ISO A2 codes) and rebuilds
|
||||
both files to cover it.
|
||||
|
||||
There's no standalone per-country file to individually delete —
|
||||
`offline.pmtiles`/`terrain.pmtiles` are always one combined extract for
|
||||
the *whole current selection*. "Remove a country" and "refresh a
|
||||
country" are both just "change the selection, then Apply" (unticking a
|
||||
country and applying rebuilds without it; nothing to separately
|
||||
delete). Every Apply re-extracts the entire selection from scratch, not
|
||||
just what changed — fine given extraction is fast (byte-range requests
|
||||
against remote archives, not per-tile fetches), but worth revisiting if
|
||||
someone selects half of Europe at once and finds it slow.
|
||||
|
||||
This wraps exactly the manual `go-pmtiles` workflows documented above
|
||||
(`lib/maptool/pmtiles_builder.rb`) — the server needs the
|
||||
https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI installed and
|
||||
on `PATH` (override with the `PMTILES_BIN` env var if it lives
|
||||
somewhere else). Fetches Mapterhorn's own small JSON manifest to work
|
||||
out which regional chunk(s), if any, intersect the bbox, `extract`s
|
||||
each (plus `planet.pmtiles`, always), then `merge`s them if there was
|
||||
more than one — typically seconds, not the tens of minutes an earlier
|
||||
version of this (fetching every individual Terrarium tile itself) used
|
||||
to take for a real-sized region.
|
||||
somewhere else), using `--region=<geojson>` (a `FeatureCollection` of
|
||||
the selected countries — go-pmtiles treats multiple features as their
|
||||
union already, no union/dissolve math needed on this side) rather than
|
||||
`--bbox=` for precise, per-country extraction. terrain.pmtiles extracts
|
||||
`planet.pmtiles` (always) plus whichever Mapterhorn regional chunks
|
||||
intersect the selection, `merge`s them if there was more than one —
|
||||
always safe regardless of how many/which countries are selected, since
|
||||
each chunk is Mapterhorn's own pre-partitioned, non-overlapping source
|
||||
file (extracting the *same* source once per country and merging those,
|
||||
by contrast, would fail — adjacent countries' region-extracts share
|
||||
border tiles, since extraction copies whole tiles rather than clipping
|
||||
vector content to the polygon, and `pmtiles merge` refuses any
|
||||
overlapping tile id between inputs).
|
||||
|
||||
A build runs in the background (`POST /api/offline-tiles/build` /
|
||||
`/api/terrain-tiles/build`, polled via the matching `*/build-status`
|
||||
route) — only one of each kind at a time; starting a second while one's
|
||||
running gets a 409.
|
||||
A build runs in the background (`GET /api/offline-regions` to list/see
|
||||
the current selection, `POST` to change it, then `POST
|
||||
/api/offline-tiles/build` / `/api/terrain-tiles/build`, polled via the
|
||||
matching `*/build-status` route) — only one of each kind at a time;
|
||||
starting a second while one's running gets a 409, as does starting
|
||||
either with nothing selected yet.
|
||||
|
||||
== Camper-relevant POIs (Overpass)
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"name":"Albania","iso_a2":"AL","iso_a3":"ALB"},"geometry":{"type":"Polygon","coordinates":[[[21.02004,40.842727],[20.99999,40.580004],[20.674997,40.435],[20.615,40.110007],[20.150016,39.624998],[19.98,39.694993],[19.960002,39.915006],[19.406082,40.250773],[19.319059,40.72723],[19.40355,41.409566],[19.540027,41.719986],[19.371769,41.877548],[19.371768,41.877551],[19.304486,42.195745],[19.738051,42.688247],[19.801613,42.500093],[20.0707,42.58863],[20.283755,42.32026],[20.52295,42.21787],[20.590247,41.855409],[20.590247,41.855404],[20.463175,41.515089],[20.605182,41.086226],[21.02004,40.842727]]]}},{"type":"Feature","properties":{"name":"Austria","iso_a2":"AT","iso_a3":"AUT"},"geometry":{"type":"Polygon","coordinates":[[[16.979667,48.123497],[16.903754,47.714866],[16.340584,47.712902],[16.534268,47.496171],[16.202298,46.852386],[16.011664,46.683611],[15.137092,46.658703],[14.632472,46.431817],[13.806475,46.509306],[12.376485,46.767559],[12.153088,47.115393],[11.164828,46.941579],[11.048556,46.751359],[10.442701,46.893546],[9.932448,46.920728],[9.47997,47.10281],[9.632932,47.347601],[9.594226,47.525058],[9.896068,47.580197],[10.402084,47.302488],[10.544504,47.566399],[11.426414,47.523766],[12.141357,47.703083],[12.62076,47.672388],[12.932627,47.467646],[13.025851,47.637584],[12.884103,48.289146],[13.243357,48.416115],[13.595946,48.877172],[14.338898,48.555305],[14.901447,48.964402],[15.253416,49.039074],[16.029647,48.733899],[16.499283,48.785808],[16.960288,48.596982],[16.879983,48.470013],[16.979667,48.123497]]]}},{"type":"Feature","properties":{"name":"Belarus","iso_a2":"BY","iso_a3":"BLR"},"geometry":{"type":"Polygon","coordinates":[[[28.176709,56.16913],[29.229513,55.918344],[29.371572,55.670091],[29.896294,55.789463],[30.873909,55.550976],[30.971836,55.081548],[30.757534,54.811771],[31.384472,54.157056],[31.791424,53.974639],[31.731273,53.794029],[32.405599,53.618045],[32.693643,53.351421],[32.304519,53.132726],[31.49764,53.16743],[31.305201,53.073996],[31.540018,52.742052],[31.78597,52.10168],[31.785992,52.101678],[30.927549,52.042353],[30.619454,51.822806],[30.555117,51.319503],[30.157364,51.416138],[29.254938,51.368234],[28.992835,51.602044],[28.617613,51.427714],[28.241615,51.572227],[27.454066,51.592303],[26.337959,51.832289],[25.327788,51.910656],[24.553106,51.888461],[24.005078,51.617444],[23.527071,51.578454],[23.508002,52.023647],[23.199494,52.486977],[23.799199,52.691099],[23.804935,53.089731],[23.527536,53.470122],[23.484128,53.912498],[24.450684,53.905702],[25.536354,54.282423],[25.768433,54.846963],[26.588279,55.167176],[26.494331,55.615107],[27.10246,55.783314],[28.176709,56.16913]]]}},{"type":"Feature","properties":{"name":"Belgium","iso_a2":"BE","iso_a3":"BEL"},"geometry":{"type":"Polygon","coordinates":[[[6.156658,50.803721],[6.043073,50.128052],[5.782417,50.090328],[5.674052,49.529484],[4.799222,49.985373],[4.286023,49.907497],[3.588184,50.378992],[3.123252,50.780363],[2.658422,50.796848],[2.513573,51.148506],[3.314971,51.345781],[3.315011,51.345777],[3.314971,51.345755],[4.047071,51.267259],[4.973991,51.475024],[5.606976,51.037298],[6.156658,50.803721]]]}},{"type":"Feature","properties":{"name":"Bosnia and Herz.","iso_a2":"BA","iso_a3":"BIH"},"geometry":{"type":"Polygon","coordinates":[[[18.56,42.65],[17.674922,43.028563],[17.297373,43.446341],[16.916156,43.667722],[16.456443,44.04124],[16.23966,44.351143],[15.750026,44.818712],[15.959367,45.233777],[16.318157,45.004127],[16.534939,45.211608],[17.002146,45.233777],[17.861783,45.06774],[18.553214,45.08159],[19.005485,44.860234],[19.00548,44.86023],[19.36803,44.863],[19.11761,44.42307],[19.59976,44.03847],[19.454,43.5681],[19.21852,43.52384],[19.03165,43.43253],[18.70648,43.20011],[18.56,42.65]]]}},{"type":"Feature","properties":{"name":"Bulgaria","iso_a2":"BG","iso_a3":"BGR"},"geometry":{"type":"Polygon","coordinates":[[[22.65715,44.234923],[22.944832,43.823785],[23.332302,43.897011],[24.100679,43.741051],[25.569272,43.688445],[26.065159,43.943494],[27.2424,44.175986],[27.970107,43.812468],[28.558081,43.707462],[28.039095,43.293172],[27.673898,42.577892],[27.99672,42.007359],[27.135739,42.141485],[26.117042,41.826905],[26.106138,41.328899],[25.197201,41.234486],[24.492645,41.583896],[23.692074,41.309081],[22.952377,41.337994],[22.881374,41.999297],[22.380526,42.32026],[22.545012,42.461362],[22.436595,42.580321],[22.604801,42.898519],[22.986019,43.211161],[22.500157,43.642814],[22.410446,44.008063],[22.65715,44.234923]]]}},{"type":"Feature","properties":{"name":"Croatia","iso_a2":"HR","iso_a3":"HRV"},"geometry":{"type":"Polygon","coordinates":[[[16.564808,46.503751],[16.882515,46.380632],[17.630066,45.951769],[18.456062,45.759481],[18.829825,45.908872],[19.072769,45.521511],[19.390476,45.236516],[19.005485,44.860234],[18.553214,45.08159],[17.861783,45.06774],[17.002146,45.233777],[16.534939,45.211608],[16.318157,45.004127],[15.959367,45.233777],[15.750026,44.818712],[Line truncated
|
||||
+122
-28
@@ -6,17 +6,28 @@ require "tmpdir"
|
||||
require "fileutils"
|
||||
|
||||
module Maptool
|
||||
# Regenerates data/offline.pmtiles and data/terrain.pmtiles for a given
|
||||
# bbox by shelling out to go-pmtiles — the same tool docs/development.adoc
|
||||
# already documents running by hand. No job queue: each build runs on its
|
||||
# own background Thread and reports into a Mutex-guarded status Hash a
|
||||
# route can poll, the same shape app.rb's own GNSS poller (GNSS_LATEST/
|
||||
# GNSS_LATEST_MUTEX) already uses.
|
||||
# Regenerates data/offline.pmtiles and data/terrain.pmtiles for the
|
||||
# currently *selected* regions (data/offline_regions.json, an array of
|
||||
# Natural Earth iso_a2 country codes — see available_regions/
|
||||
# selected_regions/set_selected_regions below) by shelling out to
|
||||
# go-pmtiles — the same tool docs/development.adoc already documents
|
||||
# running by hand. No job queue: each build runs on its own background
|
||||
# Thread and reports into a Mutex-guarded status Hash a route can poll,
|
||||
# the same shape app.rb's own GNSS poller (GNSS_LATEST/GNSS_LATEST_MUTEX)
|
||||
# already uses.
|
||||
module PmtilesBuilder
|
||||
class BuildError < StandardError; end
|
||||
|
||||
# Natural Earth (public domain, no attribution required) 110m admin-0
|
||||
# countries, filtered to Europe and trimmed to name/iso_a2/iso_a3 —
|
||||
# see the plan this was built from for the exact filter. 110m is
|
||||
# deliberately coarse (a km or so of border precision, not
|
||||
# survey-grade) — fine for "which tiles to grab," a same-schema swap
|
||||
# to Natural Earth's 50m/10m cut if that ever needs to be tighter.
|
||||
REGIONS_FILE = File.join(__dir__, "data", "europe_countries.geojson")
|
||||
|
||||
# Protomaps' own daily full-planet basemap build index — extract_offline
|
||||
# below only pulls the HTTP byte ranges covering the requested bbox
|
||||
# below only pulls the HTTP byte ranges covering the requested region
|
||||
# (go-pmtiles's `extract` supports a remote source over range requests),
|
||||
# never the whole ~120GB file.
|
||||
PROTOMAPS_BUILDS_INDEX = "https://build-metadata.protomaps.dev/builds.json"
|
||||
@@ -25,7 +36,7 @@ module Maptool
|
||||
# Mapterhorn (a Protomaps-affiliated project) publishes Terrarium-
|
||||
# encoded terrain as pre-built PMTiles archives: one global
|
||||
# planet.pmtiles (z0-12) plus ~450 regional chunks (z13 up to a
|
||||
# per-region max, 13-18) — same "extract only the bbox's byte ranges
|
||||
# per-region max, 13-18) — same "extract only the region's byte ranges
|
||||
# from a remote archive" story as offline.pmtiles above, so no more
|
||||
# fetching individual tiles or building an MBTiles file ourselves the
|
||||
# way an earlier version of this file did.
|
||||
@@ -59,25 +70,97 @@ module Maptool
|
||||
TERRAIN_MUTEX.synchronize { TERRAIN_STATUS.replace(state: "idle") }
|
||||
end
|
||||
|
||||
# bounds: [south, west, north, east]. Returns once the background Thread
|
||||
# has started, not once the build finishes — raises BuildError (the
|
||||
# caller turns this into a 409) if one's already running.
|
||||
# --- region selection ---
|
||||
|
||||
def self.available_regions
|
||||
region_features.map { |f| { code: f["properties"]["iso_a2"], name: f["properties"]["name"] } }.sort_by { |r| r[:name] }
|
||||
end
|
||||
|
||||
def self.selected_regions(data_dir)
|
||||
path = selected_regions_path(data_dir)
|
||||
return [] unless File.exist?(path)
|
||||
|
||||
JSON.parse(File.read(path))
|
||||
rescue JSON::ParserError
|
||||
[]
|
||||
end
|
||||
|
||||
def self.set_selected_regions(data_dir, codes)
|
||||
unknown = codes - available_regions.map { |r| r[:code] }
|
||||
raise BuildError, "unknown region code(s): #{unknown.join(", ")}" unless unknown.empty?
|
||||
|
||||
File.write(selected_regions_path(data_dir), JSON.generate(codes))
|
||||
end
|
||||
|
||||
def self.selected_regions_path(data_dir)
|
||||
File.join(data_dir, "offline_regions.json")
|
||||
end
|
||||
private_class_method :selected_regions_path
|
||||
|
||||
def self.region_features
|
||||
@region_features ||= JSON.parse(File.read(REGIONS_FILE))["features"]
|
||||
end
|
||||
private_class_method :region_features
|
||||
|
||||
# Writes the FeatureCollection of just the selected countries into
|
||||
# `dir` — this *is* the file go-pmtiles' `--region=` reads. No
|
||||
# union/dissolve needed: go-pmtiles treats a FeatureCollection's
|
||||
# features as one combined MultiPolygon on its own (confirmed against
|
||||
# its own source, pmtiles/region.go's UnmarshalRegion). Also returns a
|
||||
# plain aggregate [south,west,north,east] (min/max over every ring
|
||||
# coordinate, no geometry library) used only for the Mapterhorn
|
||||
# chunk-intersection check below — approximate is fine there, the
|
||||
# `--region=` file is what does the real precise selection.
|
||||
def self.region_geojson_and_bbox(codes, dir)
|
||||
selected = region_features.select { |f| codes.include?(f["properties"]["iso_a2"]) }
|
||||
raise BuildError, "no matching regions for #{codes.join(", ")}" if selected.empty?
|
||||
|
||||
path = File.join(dir, "region.geojson")
|
||||
File.write(path, JSON.generate(type: "FeatureCollection", features: selected))
|
||||
|
||||
coords = selected.flat_map { |f| ring_coordinates(f["geometry"]) }
|
||||
lons = coords.map { |lng, _lat| lng }
|
||||
lats = coords.map { |_lng, lat| lat }
|
||||
[path, [lats.min, lons.min, lats.max, lons.max]]
|
||||
end
|
||||
private_class_method :region_geojson_and_bbox
|
||||
|
||||
def self.ring_coordinates(geometry)
|
||||
case geometry["type"]
|
||||
when "Polygon" then geometry["coordinates"].flatten(1)
|
||||
when "MultiPolygon" then geometry["coordinates"].flatten(2)
|
||||
else []
|
||||
end
|
||||
end
|
||||
private_class_method :ring_coordinates
|
||||
|
||||
# --- builds ---
|
||||
|
||||
# Returns once the background Thread has started, not once the build
|
||||
# finishes — raises BuildError (the caller turns this into an error
|
||||
# response) if nothing's selected yet, or one's already running.
|
||||
#
|
||||
# fetch_build_url exists as an injectable seam so tests can exercise the
|
||||
# rest of this (temp path, go-pmtiles invocation, atomic rename) without
|
||||
# a real network call — same reasoning test_helper.rb already gives for
|
||||
# keeping /api/plugins route tests off the real Overpass API.
|
||||
def self.build_offline(bounds, data_dir:, fetch_build_url: method(:latest_protomaps_build_url))
|
||||
def self.build_offline(data_dir:, fetch_build_url: method(:latest_protomaps_build_url))
|
||||
codes = selected_regions(data_dir)
|
||||
raise BuildError, "select at least one region first" if codes.empty?
|
||||
|
||||
begin_run!(OFFLINE_MUTEX, OFFLINE_STATUS)
|
||||
Thread.new { run_job(OFFLINE_MUTEX, OFFLINE_STATUS) { extract_offline(bounds, data_dir, fetch_build_url) } }
|
||||
Thread.new { run_job(OFFLINE_MUTEX, OFFLINE_STATUS) { extract_offline(codes, data_dir, fetch_build_url) } }
|
||||
end
|
||||
|
||||
# fetch_manifest: same injectable-seam reasoning as fetch_build_url
|
||||
# above, for the one Mapterhorn manifest lookup this needs instead of
|
||||
# the Protomaps build index.
|
||||
def self.build_terrain(bounds, data_dir:, fetch_manifest: method(:fetch_mapterhorn_manifest))
|
||||
def self.build_terrain(data_dir:, fetch_manifest: method(:fetch_mapterhorn_manifest))
|
||||
codes = selected_regions(data_dir)
|
||||
raise BuildError, "select at least one region first" if codes.empty?
|
||||
|
||||
begin_run!(TERRAIN_MUTEX, TERRAIN_STATUS)
|
||||
Thread.new { run_job(TERRAIN_MUTEX, TERRAIN_STATUS) { build_terrain_pmtiles(bounds, data_dir, fetch_manifest) } }
|
||||
Thread.new { run_job(TERRAIN_MUTEX, TERRAIN_STATUS) { build_terrain_pmtiles(codes, data_dir, fetch_manifest) } }
|
||||
end
|
||||
|
||||
def self.begin_run!(mutex, status)
|
||||
@@ -98,14 +181,16 @@ module Maptool
|
||||
|
||||
# --- offline.pmtiles ---
|
||||
|
||||
def self.extract_offline(bounds, data_dir, fetch_build_url)
|
||||
def self.extract_offline(codes, data_dir, fetch_build_url)
|
||||
source = fetch_build_url.call
|
||||
south, west, north, east = bounds
|
||||
scratch_dir = Dir.mktmpdir("maptool-offline")
|
||||
region_path, = region_geojson_and_bbox(codes, scratch_dir)
|
||||
tmp_path = File.join(data_dir, "offline.pmtiles.tmp")
|
||||
|
||||
run_pmtiles!("extract", source, tmp_path, "--bbox=#{west},#{south},#{east},#{north}", "--maxzoom=15")
|
||||
run_pmtiles!("extract", source, tmp_path, "--region=#{region_path}", "--maxzoom=15")
|
||||
File.rename(tmp_path, File.join(data_dir, "offline.pmtiles"))
|
||||
ensure
|
||||
FileUtils.remove_entry(scratch_dir) if scratch_dir && Dir.exist?(scratch_dir)
|
||||
File.delete(tmp_path) if tmp_path && File.exist?(tmp_path)
|
||||
end
|
||||
private_class_method :extract_offline
|
||||
@@ -126,25 +211,34 @@ module Maptool
|
||||
# --- terrain.pmtiles ---
|
||||
|
||||
# planet.pmtiles (z0-12, always) plus whichever regional chunks
|
||||
# intersect the bbox (z13+, zero or more — most bboxes hit exactly one,
|
||||
# a bbox straddling a chunk boundary hits two) are each `extract`ed on
|
||||
# their own, then combined with `pmtiles merge` if there was more than
|
||||
# one (planet + a chunk are disjoint by zoom level, so this is always a
|
||||
# valid merge; go-pmtiles requires clustered, matching-tile-type inputs,
|
||||
# which every Mapterhorn PMTiles archive already satisfies).
|
||||
def self.build_terrain_pmtiles(bounds, data_dir, fetch_manifest)
|
||||
south, west, north, east = bounds
|
||||
# intersect the selection's aggregate bbox (z13+, zero or more) are
|
||||
# each `extract`ed on their own using the *same* --region= file, then
|
||||
# combined with `pmtiles merge` if there was more than one. Always
|
||||
# safe to merge regardless of how many countries are selected or
|
||||
# whether they're adjacent: each Mapterhorn chunk is its own
|
||||
# pre-partitioned, non-overlapping source grid cell, so two different
|
||||
# chunks' extracts can never share a tile id — the only thing `merge`
|
||||
# refuses (see pmtiles/merge.go's own disjointness check). What would
|
||||
# NOT be safe is extracting the *same* source once per country and
|
||||
# merging those — adjacent countries' region-extracts would share
|
||||
# border tiles (extraction copies whole tiles, it doesn't clip vector
|
||||
# content to the polygon) — so this always does one extract per
|
||||
# *source archive*, never one per country.
|
||||
def self.build_terrain_pmtiles(codes, data_dir, fetch_manifest)
|
||||
items = fetch_manifest.call
|
||||
planet = items.find { |item| item["name"] == MAPTERHORN_PLANET_NAME }
|
||||
raise BuildError, "mapterhorn manifest is missing #{MAPTERHORN_PLANET_NAME}" unless planet
|
||||
|
||||
scratch_dir = Dir.mktmpdir("maptool-terrain")
|
||||
region_path, bbox = region_geojson_and_bbox(codes, scratch_dir)
|
||||
south, west, north, east = bbox
|
||||
|
||||
chunks = items.select { |item| item["name"] != MAPTERHORN_PLANET_NAME && bbox_intersects?(item, south, west, north, east) }
|
||||
|
||||
scratch_dir = Dir.mktmpdir("maptool-terrain")
|
||||
tmp_path = File.join(data_dir, "terrain.pmtiles.tmp")
|
||||
extracts = [planet, *chunks].each_with_index.map do |item, i|
|
||||
extract_path = File.join(scratch_dir, "extract-#{i}.pmtiles")
|
||||
run_pmtiles!("extract", item["url"], extract_path, "--bbox=#{west},#{south},#{east},#{north}", "--maxzoom=#{TERRAIN_MAX_ZOOM}")
|
||||
run_pmtiles!("extract", item["url"], extract_path, "--region=#{region_path}", "--maxzoom=#{TERRAIN_MAX_ZOOM}")
|
||||
extract_path
|
||||
end
|
||||
|
||||
|
||||
+38
-12
@@ -401,39 +401,65 @@ html, body {
|
||||
padding: 8px 12px;
|
||||
}
|
||||
|
||||
.offline-data-panel__row + .offline-data-panel__row {
|
||||
margin-top: 10px;
|
||||
padding-top: 10px;
|
||||
border-top: 1px solid #eee;
|
||||
.offline-data-panel__regions-label {
|
||||
margin: 0 0 4px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.offline-data-panel__row-header {
|
||||
.offline-data-panel__region-list {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
max-height: 220px;
|
||||
overflow-y: auto;
|
||||
border: 1px solid #eee;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.offline-data-panel__region-list label {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 8px;
|
||||
gap: 6px;
|
||||
padding: 4px 8px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.offline-data-panel__build {
|
||||
flex-shrink: 0;
|
||||
.offline-data-panel__region-list label:hover {
|
||||
background: #f0f0f0;
|
||||
}
|
||||
|
||||
.offline-data-panel__region-status {
|
||||
padding: 6px 8px;
|
||||
color: #888;
|
||||
}
|
||||
|
||||
.offline-data-panel__apply {
|
||||
margin-top: 8px;
|
||||
width: 100%;
|
||||
border: 1px solid #ccc;
|
||||
border-radius: 4px;
|
||||
background: #fff;
|
||||
padding: 4px 8px;
|
||||
padding: 6px 8px;
|
||||
font: inherit;
|
||||
font-size: 12px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.offline-data-panel__build:hover {
|
||||
.offline-data-panel__apply:hover {
|
||||
background: #f0f0f0;
|
||||
}
|
||||
|
||||
.offline-data-panel__build:disabled {
|
||||
.offline-data-panel__apply:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.offline-data-panel__row {
|
||||
margin-top: 14px;
|
||||
padding-top: 10px;
|
||||
border-top: 1px solid #eee;
|
||||
}
|
||||
|
||||
.offline-data-panel__status {
|
||||
margin: 6px 0 0;
|
||||
color: #666;
|
||||
|
||||
+1
-1
@@ -516,7 +516,7 @@ if (isLoggedIn()) new PluginsControl(map).mount(sidebar);
|
||||
sidebar.insertAdjacentHTML("beforeend", `<div class="sidebar-group-label" title="Offline hub"></div>`);
|
||||
layerToggle.mount(sidebar);
|
||||
layerToggle.setActive("online");
|
||||
new OfflineDataControl(map).mount(sidebar);
|
||||
new OfflineDataControl().mount(sidebar);
|
||||
|
||||
sidebar.insertAdjacentHTML("beforeend", `<div class="sidebar-group-label" title="Navigation"></div>`);
|
||||
gnssControl.mount(sidebar);
|
||||
|
||||
+97
-52
@@ -1,6 +1,7 @@
|
||||
import { OFFLINE_DATA_ICON } from "/js/icons.js";
|
||||
import { notifySidePanelOpen, onlyOneSidePanelOpen } from "/js/panels.js";
|
||||
import { isLoggedIn } from "/js/auth.js";
|
||||
import { escapeHtml } from "/js/gpx.js";
|
||||
|
||||
const POLL_INTERVAL_MS = 3000;
|
||||
|
||||
@@ -16,51 +17,106 @@ function formatElapsed(ms) {
|
||||
return minutes > 0 ? `${minutes}m ${seconds}s` : `${seconds}s`;
|
||||
}
|
||||
|
||||
// "Building…" alone gave no way to tell a slow build (terrain.pmtiles can
|
||||
// take minutes — lib/maptool/pmtiles_builder.rb fetches tiles one at a
|
||||
// time) from a stuck one. Elapsed time alone answers "is it alive" for
|
||||
// both files; tile counts (only terrain.pmtiles reports these) give an
|
||||
// actual sense of how much is left.
|
||||
// "Building…" alone gave no way to tell a slow build from a stuck one —
|
||||
// elapsed time answers "is it alive" regardless of how long a given
|
||||
// selection takes.
|
||||
function formatRunning(status) {
|
||||
const elapsed = status.started_at ? ` (${formatElapsed(Date.now() - new Date(status.started_at).getTime())})` : "";
|
||||
const tiles = status.tiles_total ? ` · ${status.tiles_done}/${status.tiles_total} tiles` : "";
|
||||
return `Building…${elapsed}${tiles}`;
|
||||
return `Building…${elapsed}`;
|
||||
}
|
||||
|
||||
// One row of the Offline data panel — status text plus a "Regenerate for
|
||||
// current view" button, for either offline.pmtiles or terrain.pmtiles.
|
||||
// Both files behave identically from here (same status/build/build-status
|
||||
// endpoint shape, see lib/maptool/pmtiles_builder.rb), just against
|
||||
// different URLs/labels, so one class covers both rather than writing the
|
||||
// same fetch/poll logic twice.
|
||||
// The checklist of selectable countries (GET /api/offline-regions) plus
|
||||
// the "Apply changes" button that saves the ticked set (POST, same URL)
|
||||
// and hands it off — doesn't trigger a build itself, OfflineDataControl
|
||||
// does that across both files once the selection is saved.
|
||||
class RegionPicker {
|
||||
constructor(onApply) {
|
||||
this._onApply = onApply;
|
||||
}
|
||||
|
||||
async mount(container) {
|
||||
const section = document.createElement("div");
|
||||
section.className = "offline-data-panel__regions";
|
||||
section.innerHTML = `
|
||||
<p class="offline-data-panel__regions-label">Regions</p>
|
||||
<ul class="offline-data-panel__region-list"><li class="offline-data-panel__region-status">Loading…</li></ul>
|
||||
<button type="button" class="offline-data-panel__apply" hidden>Apply changes</button>
|
||||
`;
|
||||
this._list = section.querySelector(".offline-data-panel__region-list");
|
||||
this._applyButton = section.querySelector(".offline-data-panel__apply");
|
||||
// Viewing/ticking works for anyone; only saving needs a login — same
|
||||
// finer-grained gating measure.js/gpx.js use for their own
|
||||
// write-only affordances, rather than hiding this whole control.
|
||||
this._applyButton.hidden = !isLoggedIn();
|
||||
this._applyButton.addEventListener("click", () => this._apply());
|
||||
container.appendChild(section);
|
||||
|
||||
let data;
|
||||
try {
|
||||
data = await (await fetch("/api/offline-regions")).json();
|
||||
} catch {
|
||||
this._list.innerHTML = `<li class="offline-data-panel__region-status">Could not load the region list.</li>`;
|
||||
return;
|
||||
}
|
||||
|
||||
const selected = new Set(data.selected);
|
||||
this._list.innerHTML = data.available
|
||||
.map(
|
||||
(region) => `
|
||||
<li>
|
||||
<label>
|
||||
<input type="checkbox" value="${escapeHtml(region.code)}" ${selected.has(region.code) ? "checked" : ""}>
|
||||
${escapeHtml(region.name)}
|
||||
</label>
|
||||
</li>`,
|
||||
)
|
||||
.join("");
|
||||
}
|
||||
|
||||
async _apply() {
|
||||
const codes = [...this._list.querySelectorAll("input:checked")].map((el) => el.value);
|
||||
const body = new URLSearchParams();
|
||||
codes.forEach((code) => body.append("codes[]", code));
|
||||
|
||||
this._applyButton.disabled = true;
|
||||
try {
|
||||
const response = await fetch("/api/offline-regions", { method: "POST", body });
|
||||
const data = await response.json().catch(() => ({}));
|
||||
if (!response.ok) throw new Error(data.error || `could not save the selection: ${response.status}`);
|
||||
} catch (err) {
|
||||
window.alert(err.message);
|
||||
return;
|
||||
} finally {
|
||||
this._applyButton.disabled = false;
|
||||
}
|
||||
|
||||
this._onApply();
|
||||
}
|
||||
}
|
||||
|
||||
// One row of the Offline data panel — just a status readout for either
|
||||
// offline.pmtiles or terrain.pmtiles, both driven by the same selection
|
||||
// (RegionPicker above). Both files behave identically from here (same
|
||||
// status/build/build-status endpoint shape, see
|
||||
// lib/maptool/pmtiles_builder.rb), just against different URLs/labels, so
|
||||
// one class covers both rather than writing the same fetch/poll logic
|
||||
// twice.
|
||||
class DataFileRow {
|
||||
constructor(map, { label, statusUrl, buildUrl, buildStatusUrl }) {
|
||||
this._map = map;
|
||||
constructor({ label, statusUrl, buildUrl, buildStatusUrl }) {
|
||||
this._label = label;
|
||||
this._statusUrl = statusUrl;
|
||||
this._buildUrl = buildUrl;
|
||||
this._buildStatusUrl = buildStatusUrl;
|
||||
this._exists = false;
|
||||
}
|
||||
|
||||
mount(container) {
|
||||
const row = document.createElement("div");
|
||||
row.className = "offline-data-panel__row";
|
||||
row.innerHTML = `
|
||||
<div class="offline-data-panel__row-header">
|
||||
<strong>${this._label}</strong>
|
||||
<button type="button" class="offline-data-panel__build">Regenerate for current view</button>
|
||||
</div>
|
||||
<p class="offline-data-panel__status">Checking…</p>
|
||||
`;
|
||||
this._buildButton = row.querySelector(".offline-data-panel__build");
|
||||
this._statusEl = row.querySelector(".offline-data-panel__status");
|
||||
// Viewing status works for anyone; only triggering a build needs a
|
||||
// login — same finer-grained gating measure.js/gpx.js use for their
|
||||
// own write-only affordances, rather than hiding this whole control.
|
||||
this._buildButton.hidden = !isLoggedIn();
|
||||
this._buildButton.addEventListener("click", () => this._build());
|
||||
|
||||
container.appendChild(row);
|
||||
this._refreshStatus();
|
||||
}
|
||||
@@ -73,34 +129,21 @@ class DataFileRow {
|
||||
this._statusEl.textContent = "Could not check status.";
|
||||
return;
|
||||
}
|
||||
this._exists = data.available;
|
||||
this._statusEl.textContent = data.available
|
||||
? `${formatBytes(data.size)} · updated ${new Date(data.mtime).toLocaleString([], { dateStyle: "short", timeStyle: "short" })}`
|
||||
: "Not generated yet.";
|
||||
}
|
||||
|
||||
async _build() {
|
||||
if (this._exists && !window.confirm(`Replace the existing ${this._label.toLowerCase()} with a fresh build for the current view?`)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const bounds = this._map.getBounds();
|
||||
const body = new URLSearchParams({
|
||||
south: bounds.getSouth(),
|
||||
west: bounds.getWest(),
|
||||
north: bounds.getNorth(),
|
||||
east: bounds.getEast(),
|
||||
});
|
||||
|
||||
this._buildButton.disabled = true;
|
||||
// Called by OfflineDataControl once RegionPicker's Apply has saved a
|
||||
// new selection — not tied to a button of this row's own.
|
||||
async triggerBuild() {
|
||||
this._statusEl.textContent = "Starting…";
|
||||
try {
|
||||
const response = await fetch(this._buildUrl, { method: "POST", body });
|
||||
const response = await fetch(this._buildUrl, { method: "POST" });
|
||||
const data = await response.json().catch(() => ({}));
|
||||
if (!response.ok) throw new Error(data.error || `build failed: ${response.status}`);
|
||||
} catch (err) {
|
||||
this._statusEl.textContent = err.message;
|
||||
this._buildButton.disabled = false;
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -126,7 +169,6 @@ class DataFileRow {
|
||||
return;
|
||||
}
|
||||
|
||||
this._buildButton.disabled = false;
|
||||
if (status.state === "error") {
|
||||
this._statusEl.textContent = `Build failed: ${status.message}`;
|
||||
} else {
|
||||
@@ -137,27 +179,29 @@ class DataFileRow {
|
||||
}
|
||||
|
||||
// A sidebar panel wrapping the manual go-pmtiles workflow
|
||||
// docs/development.adoc documents — "Regenerate for current view" on
|
||||
// either file shells out to it server-side instead of an admin running it
|
||||
// by hand (see lib/maptool/pmtiles_builder.rb). No area-picker: the
|
||||
// current map viewport is the region, same convention the Overpass POI
|
||||
// plugin already uses for "give me a bbox."
|
||||
// docs/development.adoc documents — a region picker (which countries
|
||||
// should be available offline) plus, on "Apply changes", a rebuild of
|
||||
// offline.pmtiles/terrain.pmtiles covering the whole current selection
|
||||
// (see lib/maptool/pmtiles_builder.rb — there's no standalone per-country
|
||||
// file to individually delete; "remove a country" and "refresh a
|
||||
// country" are both just "change the selection, then Apply").
|
||||
export class OfflineDataControl {
|
||||
constructor(map) {
|
||||
constructor() {
|
||||
this._rows = [
|
||||
new DataFileRow(map, {
|
||||
new DataFileRow({
|
||||
label: "Offline map data",
|
||||
statusUrl: "/api/offline-tiles",
|
||||
buildUrl: "/api/offline-tiles/build",
|
||||
buildStatusUrl: "/api/offline-tiles/build-status",
|
||||
}),
|
||||
new DataFileRow(map, {
|
||||
new DataFileRow({
|
||||
label: "Offline terrain data",
|
||||
statusUrl: "/api/terrain-tiles",
|
||||
buildUrl: "/api/terrain-tiles/build",
|
||||
buildStatusUrl: "/api/terrain-tiles/build-status",
|
||||
}),
|
||||
];
|
||||
this._regionPicker = new RegionPicker(() => this._rows.forEach((row) => row.triggerBuild()));
|
||||
}
|
||||
|
||||
mount(sidebar) {
|
||||
@@ -171,6 +215,7 @@ export class OfflineDataControl {
|
||||
this._panel.querySelector(".basemap-panel__close").addEventListener("click", () => this._togglePanel(false));
|
||||
|
||||
const body = this._panel.querySelector(".offline-data-panel__body");
|
||||
this._regionPicker.mount(body);
|
||||
this._rows.forEach((row) => row.mount(body));
|
||||
|
||||
onlyOneSidePanelOpen(this._panel, () => this._icon.classList.remove("active"));
|
||||
|
||||
+34
-12
@@ -349,41 +349,63 @@ class AppTest < Minitest::Test
|
||||
assert_equal 502, response.status
|
||||
end
|
||||
|
||||
# --- offline regions ---
|
||||
|
||||
def test_offline_regions_lists_available_and_selected
|
||||
body = json_body(request.get("/api/offline-regions"))
|
||||
assert_operator body["available"].size, :>, 0
|
||||
assert_includes body["available"], { "code" => "SK", "name" => "Slovakia" }
|
||||
assert_equal [], body["selected"]
|
||||
end
|
||||
|
||||
def test_offline_regions_post_requires_login
|
||||
assert_equal 401, request.post("/api/offline-regions").status
|
||||
end
|
||||
|
||||
def test_offline_regions_post_saves_the_selection
|
||||
response = request.post("/api/offline-regions", params: { "codes" => %w[SK AT] }, **logged_in_env)
|
||||
assert_equal 200, response.status
|
||||
assert_equal %w[SK AT], json_body(request.get("/api/offline-regions"))["selected"]
|
||||
end
|
||||
|
||||
def test_offline_regions_post_rejects_an_unknown_code
|
||||
response = request.post("/api/offline-regions", params: { "codes" => ["XX"] }, **logged_in_env)
|
||||
assert_equal 400, response.status
|
||||
end
|
||||
|
||||
# --- pmtiles regeneration ---
|
||||
#
|
||||
# The full build (network + go-pmtiles) is covered in isolation, with
|
||||
# fakes injected, by test_pmtiles_builder.rb — same "keep this offline"
|
||||
# reasoning as the rest of this file. What's route-specific and worth
|
||||
# covering here is the bbox validation and the 409-while-running guard,
|
||||
# neither of which needs a real build to actually run.
|
||||
# covering here is the empty-selection validation and the
|
||||
# 409-while-running guard, neither of which needs a real build to
|
||||
# actually run.
|
||||
|
||||
def test_offline_tiles_build_status_defaults_to_idle
|
||||
assert_equal({ "state" => "idle" }, json_body(request.get("/api/offline-tiles/build-status")))
|
||||
assert_equal({ "state" => "idle" }, json_body(request.get("/api/terrain-tiles/build-status")))
|
||||
end
|
||||
|
||||
def test_offline_tiles_build_requires_a_bbox
|
||||
def test_offline_tiles_build_is_409_with_no_selection
|
||||
response = request.post("/api/offline-tiles/build", **logged_in_env)
|
||||
assert_equal 400, response.status
|
||||
assert_equal 409, response.status
|
||||
end
|
||||
|
||||
def test_terrain_tiles_build_requires_a_bbox
|
||||
def test_terrain_tiles_build_is_409_with_no_selection
|
||||
response = request.post("/api/terrain-tiles/build", **logged_in_env)
|
||||
assert_equal 400, response.status
|
||||
assert_equal 409, response.status
|
||||
end
|
||||
|
||||
# fetch_build_url blocks on a Queue, called directly (not through the
|
||||
# route, which has no way to inject a fake) so the build is
|
||||
# deterministically still "running" when the route call below happens.
|
||||
def test_offline_tiles_build_is_409_when_one_is_already_running
|
||||
Maptool::PmtilesBuilder.set_selected_regions(Maptool::App::DATA_DIR, ["SK"])
|
||||
gate = Queue.new
|
||||
thread = Maptool::PmtilesBuilder.build_offline([1, 2, 3, 4], data_dir: Maptool::App::DATA_DIR, fetch_build_url: -> { gate.pop })
|
||||
thread = Maptool::PmtilesBuilder.build_offline(data_dir: Maptool::App::DATA_DIR, fetch_build_url: -> { gate.pop })
|
||||
|
||||
response = request.post(
|
||||
"/api/offline-tiles/build",
|
||||
params: { south: "1", west: "2", north: "3", east: "4" },
|
||||
**logged_in_env,
|
||||
)
|
||||
response = request.post("/api/offline-tiles/build", **logged_in_env)
|
||||
assert_equal 409, response.status
|
||||
ensure
|
||||
gate.push("unused")
|
||||
|
||||
@@ -66,6 +66,7 @@ module AppTestHelper
|
||||
def teardown
|
||||
FileUtils.rm_rf(Dir.glob(File.join(Maptool::App::DATA_DIR, "gpx", "*")))
|
||||
File.write(File.join(Maptool::App::DATA_DIR, "livetrack.json"), "[]")
|
||||
FileUtils.rm_f(File.join(Maptool::App::DATA_DIR, "offline_regions.json"))
|
||||
Maptool::PmtilesBuilder.reset!
|
||||
super
|
||||
end
|
||||
|
||||
@@ -11,13 +11,19 @@ require "fileutils"
|
||||
# CLI (PMTILES_BIN) stands in for the real go-pmtiles binary, logging each
|
||||
# invocation's subcommand + first arg to PMTILES_LOG so tests can assert on
|
||||
# what actually got called (e.g. "was merge invoked, and with what").
|
||||
#
|
||||
# Country codes below are real ones from the vendored
|
||||
# lib/maptool/data/europe_countries.geojson (Slovakia, Iceland, Spain) —
|
||||
# chosen so their real bboxes fall cleanly inside/outside the fake
|
||||
# manifest's chunk extents below, without needing a second fake geometry
|
||||
# source.
|
||||
class PmtilesBuilderTest < Minitest::Test
|
||||
def setup
|
||||
@data_dir = Dir.mktmpdir("maptool-pmtiles-test-data")
|
||||
@bin_dir = Dir.mktmpdir("maptool-pmtiles-test-bin")
|
||||
@log = File.join(@bin_dir, "pmtiles.log")
|
||||
|
||||
# Output path: $3 for `extract SOURCE OUTPUT --bbox=... --maxzoom=...`
|
||||
# Output path: $3 for `extract SOURCE OUTPUT --region=... --maxzoom=...`
|
||||
# and `convert INPUT OUTPUT` alike; the *last* arg for `merge INPUT_1
|
||||
# [INPUT_2 ...] OUTPUT`, since its input count varies.
|
||||
@pmtiles_ok = File.join(@bin_dir, "pmtiles-ok")
|
||||
@@ -57,15 +63,17 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
File.exist?(@log) ? File.readlines(@log, chomp: true) : []
|
||||
end
|
||||
|
||||
# planet.pmtiles always covers the whole world (z0-12); chunk-a/chunk-b
|
||||
# are disjoint small regions (z13+), matching Mapterhorn's real shape
|
||||
# (see download_urls.json) closely enough to exercise bbox_intersects?
|
||||
# and the extract-then-maybe-merge logic without a real manifest fetch.
|
||||
def select_regions(*codes)
|
||||
Maptool::PmtilesBuilder.set_selected_regions(@data_dir, codes)
|
||||
end
|
||||
|
||||
# chunk-a covers Slovakia's real extent, chunk-b covers Iceland's —
|
||||
# Spain falls in neither, for the "no chunk intersects" case.
|
||||
def fake_manifest
|
||||
[
|
||||
{ "name" => "planet.pmtiles", "url" => "https://example.com/planet.pmtiles", "min_lon" => -180.0, "min_lat" => -85.0, "max_lon" => 180.0, "max_lat" => 85.0, "min_zoom" => 0, "max_zoom" => 12 },
|
||||
{ "name" => "chunk-a.pmtiles", "url" => "https://example.com/chunk-a.pmtiles", "min_lon" => 17.0, "min_lat" => 48.0, "max_lon" => 17.5, "max_lat" => 48.5, "min_zoom" => 13, "max_zoom" => 15 },
|
||||
{ "name" => "chunk-b.pmtiles", "url" => "https://example.com/chunk-b.pmtiles", "min_lon" => 20.0, "min_lat" => 50.0, "max_lon" => 20.5, "max_lat" => 50.5, "min_zoom" => 13, "max_zoom" => 15 },
|
||||
{ "name" => "chunk-a.pmtiles", "url" => "https://example.com/chunk-a.pmtiles", "min_lon" => 16.0, "min_lat" => 47.0, "max_lon" => 23.0, "max_lat" => 50.0, "min_zoom" => 13, "max_zoom" => 15 },
|
||||
{ "name" => "chunk-b.pmtiles", "url" => "https://example.com/chunk-b.pmtiles", "min_lon" => -25.0, "min_lat" => 63.0, "max_lon" => -13.0, "max_lat" => 67.0, "min_zoom" => 13, "max_zoom" => 15 },
|
||||
]
|
||||
end
|
||||
|
||||
@@ -74,27 +82,51 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
assert_equal({ state: "idle" }, Maptool::PmtilesBuilder.terrain_status)
|
||||
end
|
||||
|
||||
# --- region selection ---
|
||||
|
||||
def test_available_regions_includes_known_countries
|
||||
codes = Maptool::PmtilesBuilder.available_regions.map { |r| r[:code] }
|
||||
assert_includes codes, "SK"
|
||||
assert_includes codes, "IS"
|
||||
end
|
||||
|
||||
def test_selected_regions_defaults_to_empty
|
||||
assert_empty Maptool::PmtilesBuilder.selected_regions(@data_dir)
|
||||
end
|
||||
|
||||
def test_set_selected_regions_round_trips
|
||||
select_regions("SK", "AT")
|
||||
assert_equal ["SK", "AT"], Maptool::PmtilesBuilder.selected_regions(@data_dir)
|
||||
end
|
||||
|
||||
def test_set_selected_regions_rejects_an_unknown_code
|
||||
assert_raises(Maptool::PmtilesBuilder::BuildError) { select_regions("SK", "XX") }
|
||||
end
|
||||
|
||||
# --- offline.pmtiles ---
|
||||
|
||||
def test_build_offline_raises_with_no_selection
|
||||
assert_raises(Maptool::PmtilesBuilder::BuildError) do
|
||||
Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "unused" })
|
||||
end
|
||||
end
|
||||
|
||||
def test_build_offline_writes_the_file_and_reports_done
|
||||
select_regions("SK")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
thread = Maptool::PmtilesBuilder.build_offline(
|
||||
[1, 2, 3, 4],
|
||||
data_dir: @data_dir,
|
||||
fetch_build_url: -> { "https://example.com/fake.pmtiles" },
|
||||
)
|
||||
thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "https://example.com/fake.pmtiles" })
|
||||
thread.join
|
||||
|
||||
assert_path_exists File.join(@data_dir, "offline.pmtiles")
|
||||
refute_path_exists File.join(@data_dir, "offline.pmtiles.tmp")
|
||||
assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state]
|
||||
assert_equal ["extract https://example.com/fake.pmtiles"], log_lines
|
||||
end
|
||||
|
||||
def test_build_offline_reports_an_error_status_on_non_zero_exit
|
||||
select_regions("SK")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_fail
|
||||
thread = Maptool::PmtilesBuilder.build_offline(
|
||||
[1, 2, 3, 4],
|
||||
data_dir: @data_dir,
|
||||
fetch_build_url: -> { "https://example.com/fake.pmtiles" },
|
||||
)
|
||||
thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "https://example.com/fake.pmtiles" })
|
||||
thread.join
|
||||
|
||||
refute_path_exists File.join(@data_dir, "offline.pmtiles")
|
||||
@@ -104,12 +136,9 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
end
|
||||
|
||||
def test_build_offline_reports_a_clear_error_when_the_binary_is_missing
|
||||
select_regions("SK")
|
||||
ENV["PMTILES_BIN"] = File.join(@bin_dir, "does-not-exist")
|
||||
thread = Maptool::PmtilesBuilder.build_offline(
|
||||
[1, 2, 3, 4],
|
||||
data_dir: @data_dir,
|
||||
fetch_build_url: -> { "https://example.com/fake.pmtiles" },
|
||||
)
|
||||
thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "https://example.com/fake.pmtiles" })
|
||||
thread.join
|
||||
|
||||
status = Maptool::PmtilesBuilder.offline_status
|
||||
@@ -121,23 +150,33 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
# still "running" when the second call happens, rather than racing a real
|
||||
# one to see which state wins.
|
||||
def test_build_offline_raises_when_one_is_already_running
|
||||
select_regions("SK")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
gate = Queue.new
|
||||
thread = Maptool::PmtilesBuilder.build_offline([1, 2, 3, 4], data_dir: @data_dir, fetch_build_url: -> { gate.pop })
|
||||
thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { gate.pop })
|
||||
|
||||
assert_raises(Maptool::PmtilesBuilder::BuildError) do
|
||||
Maptool::PmtilesBuilder.build_offline([1, 2, 3, 4], data_dir: @data_dir, fetch_build_url: -> { "unused" })
|
||||
Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "unused" })
|
||||
end
|
||||
ensure
|
||||
gate.push("https://example.com/fake.pmtiles")
|
||||
thread&.join
|
||||
end
|
||||
|
||||
# A bbox nowhere near chunk-a/chunk-b — only planet.pmtiles matches, so
|
||||
# --- terrain.pmtiles ---
|
||||
|
||||
def test_build_terrain_raises_with_no_selection
|
||||
assert_raises(Maptool::PmtilesBuilder::BuildError) do
|
||||
Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
end
|
||||
end
|
||||
|
||||
# Spain intersects neither fake chunk — only planet.pmtiles matches, so
|
||||
# this should be a single extract straight to the final path, no merge.
|
||||
def test_build_terrain_extracts_only_planet_when_no_chunk_intersects
|
||||
select_regions("ES")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread.join
|
||||
|
||||
assert_path_exists File.join(@data_dir, "terrain.pmtiles")
|
||||
@@ -145,11 +184,12 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
assert_equal ["extract https://example.com/planet.pmtiles"], log_lines
|
||||
end
|
||||
|
||||
# A bbox inside chunk-a's extent — planet.pmtiles (always) plus that one
|
||||
# chunk get extracted, then merged.
|
||||
# Slovakia intersects chunk-a only — planet.pmtiles (always) plus that
|
||||
# one chunk get extracted, then merged.
|
||||
def test_build_terrain_extracts_and_merges_a_single_intersecting_chunk
|
||||
select_regions("SK")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
thread = Maptool::PmtilesBuilder.build_terrain([48.1, 17.1, 48.2, 17.2], data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread.join
|
||||
|
||||
assert_path_exists File.join(@data_dir, "terrain.pmtiles")
|
||||
@@ -160,11 +200,12 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
assert_equal 1, merged.size
|
||||
end
|
||||
|
||||
# A bbox spanning both chunk-a and chunk-b — planet + both chunks, one
|
||||
# 3-way merge.
|
||||
# Slovakia + Iceland together intersect both fake chunks — planet + both
|
||||
# chunks, one 3-way merge.
|
||||
def test_build_terrain_merges_multiple_intersecting_chunks
|
||||
select_regions("SK", "IS")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
thread = Maptool::PmtilesBuilder.build_terrain([47.9, 16.9, 50.6, 20.6], data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread.join
|
||||
|
||||
assert_equal "done", Maptool::PmtilesBuilder.terrain_status[:state]
|
||||
@@ -174,8 +215,9 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
end
|
||||
|
||||
def test_build_terrain_reports_a_clear_error_when_the_manifest_is_missing_planet
|
||||
select_regions("ES")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { fake_manifest.reject { |i| i["name"] == "planet.pmtiles" } })
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest.reject { |i| i["name"] == "planet.pmtiles" } })
|
||||
thread.join
|
||||
|
||||
status = Maptool::PmtilesBuilder.terrain_status
|
||||
@@ -184,8 +226,9 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
end
|
||||
|
||||
def test_build_terrain_reports_an_error_status_on_non_zero_exit
|
||||
select_regions("ES")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_fail
|
||||
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
|
||||
thread.join
|
||||
|
||||
refute_path_exists File.join(@data_dir, "terrain.pmtiles")
|
||||
@@ -195,12 +238,13 @@ class PmtilesBuilderTest < Minitest::Test
|
||||
end
|
||||
|
||||
def test_build_terrain_raises_when_one_is_already_running
|
||||
select_regions("ES")
|
||||
ENV["PMTILES_BIN"] = @pmtiles_ok
|
||||
gate = Queue.new
|
||||
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { gate.pop })
|
||||
thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { gate.pop })
|
||||
|
||||
assert_raises(Maptool::PmtilesBuilder::BuildError) do
|
||||
Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { "unused" })
|
||||
Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { "unused" })
|
||||
end
|
||||
ensure
|
||||
gate.push(fake_manifest)
|
||||
|
||||
Reference in new issue
Block a user