Add a country-based region picker for offline data
CI / check (push) Successful in 17s
CI / deploy (push) Successful in 12s

Replaces "regenerate for the current viewport" with what Locus/OsmAnd-
style apps give you: tick which countries you want offline (a Natural
Earth-derived checklist, public domain, vendored), Apply rebuilds
offline.pmtiles/terrain.pmtiles to cover the whole selection at once.
Multiple regions can be offline simultaneously (e.g. Slovakia + Austria
for a trip that crosses the border) — the selection itself is the
source of truth (data/offline_regions.json), not a set of per-country
files, so "remove a country" and "refresh a country" are both just
"change the selection, then Apply."

Confirmed against go-pmtiles' own source (not just its docs) before
committing to this shape: `extract --region=<geojson>` treats a
FeatureCollection's features as their union already (no dissolve/union
math needed on our side — pmtiles/region.go's UnmarshalRegion just
concatenates every feature's rings), and `merge` hard-errors on any
overlapping tile id between inputs (merge.go's prepareInputs). That
second fact rules out "extract each country separately, merge them
all" — adjacent countries' region-extracts would share border tiles,
since extraction copies whole tiles rather than clipping vector
content to the polygon. The actual fix: extract per *source archive*
(the Protomaps build, each intersecting Mapterhorn chunk), each call
covering the whole current selection via one --region= file — the
terrain pipeline already extracted-then-merged per source this way, it
just needed a multi-country region instead of a single bbox.
This commit is contained in:
randy committed 2026-09-11 12:55:56 +02:00
1 parent f89dea5805
commit 53c9a82af2
10 files changed
+441 -169

No files matched your search

+30 -13
View File
@@ -223,18 +223,38 @@ module Maptool
send_file OFFLINE_PMTILES_PATH, type: "application/octet-stream" send_file OFFLINE_PMTILES_PATH, type: "application/octet-stream"
end end
# Regenerates offline.pmtiles for the given bbox (the current map # Lists the selectable countries and which of them are currently
# viewport, same "visible area is the area" convention as the Overpass # selected (data/offline_regions.json) — read-only, no login needed,
# POI plugin) by extracting it from Protomaps' own daily full-planet # same tier as the plain availability checks above.
# basemap build — see lib/maptool/pmtiles_builder.rb. Runs in the get "/api/offline-regions" do
# background; poll build-status below for progress. json available: PmtilesBuilder.available_regions, selected: PmtilesBuilder.selected_regions(DATA_DIR)
post "/api/offline-tiles/build" do end
# Just updates the selection — does not itself start a build, so
# ticking several countries doesn't fire off several rebuilds. The
# frontend follows this with the two build routes below once the user
# is done choosing.
post "/api/offline-regions" do
require_login! require_login!
bbox = %w[south west north east].map { |key| params[key].to_s } codes = Array(params[:codes])
halt 400, json(error: "south, west, north, east are required") if bbox.any?(&:empty?)
begin begin
PmtilesBuilder.build_offline(bbox.map(&:to_f), data_dir: DATA_DIR) PmtilesBuilder.set_selected_regions(DATA_DIR, codes)
rescue PmtilesBuilder::BuildError => e
halt 400, json(error: e.message)
end
json ok: true
end
# Regenerates offline.pmtiles for the currently selected regions (see
# POST /api/offline-regions above) by extracting them from Protomaps'
# own daily full-planet basemap build — see
# lib/maptool/pmtiles_builder.rb. Runs in the background; poll
# build-status below for progress.
post "/api/offline-tiles/build" do
require_login!
begin
PmtilesBuilder.build_offline(data_dir: DATA_DIR)
rescue PmtilesBuilder::BuildError => e rescue PmtilesBuilder::BuildError => e
halt 409, json(error: e.message) halt 409, json(error: e.message)
end end
@@ -260,11 +280,8 @@ module Maptool
post "/api/terrain-tiles/build" do post "/api/terrain-tiles/build" do
require_login! require_login!
bbox = %w[south west north east].map { |key| params[key].to_s }
halt 400, json(error: "south, west, north, east are required") if bbox.any?(&:empty?)
begin begin
PmtilesBuilder.build_terrain(bbox.map(&:to_f), data_dir: DATA_DIR) PmtilesBuilder.build_terrain(data_dir: DATA_DIR)
rescue PmtilesBuilder::BuildError => e rescue PmtilesBuilder::BuildError => e
halt 409, json(error: e.message) halt 409, json(error: e.message)
end end
+37 -15
View File
@@ -85,25 +85,47 @@ size rather than assuming one, so this isn't a problem either way.
A logged-in user can also generate both files above from the app A logged-in user can also generate both files above from the app
itself, no local tooling needed — the sidebar's "Offline data" icon itself, no local tooling needed — the sidebar's "Offline data" icon
(Offline hub group) shows each file's status (size, last updated) and (Offline hub group) shows a checklist of European countries
a "Regenerate for current view" button. The region is always the (`lib/maptool/data/europe_countries.geojson`, Natural Earth 110m
current map viewport, same "no area-picker, the visible area is the admin-0, public domain) plus each file's status (size, last updated).
area" convention the Overpass POI plugin already uses below. This Ticking countries and clicking "Apply changes" saves the selection
wraps exactly the manual `go-pmtiles` workflows documented above (`data/offline_regions.json`, an array of ISO A2 codes) and rebuilds
both files to cover it.
There's no standalone per-country file to individually delete —
`offline.pmtiles`/`terrain.pmtiles` are always one combined extract for
the *whole current selection*. "Remove a country" and "refresh a
country" are both just "change the selection, then Apply" (unticking a
country and applying rebuilds without it; nothing to separately
delete). Every Apply re-extracts the entire selection from scratch, not
just what changed — fine given extraction is fast (byte-range requests
against remote archives, not per-tile fetches), but worth revisiting if
someone selects half of Europe at once and finds it slow.
This wraps exactly the manual `go-pmtiles` workflows documented above
(`lib/maptool/pmtiles_builder.rb`) — the server needs the (`lib/maptool/pmtiles_builder.rb`) — the server needs the
https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI installed and https://github.com/protomaps/go-pmtiles[go-pmtiles] CLI installed and
on `PATH` (override with the `PMTILES_BIN` env var if it lives on `PATH` (override with the `PMTILES_BIN` env var if it lives
somewhere else). Fetches Mapterhorn's own small JSON manifest to work somewhere else), using `--region=<geojson>` (a `FeatureCollection` of
out which regional chunk(s), if any, intersect the bbox, `extract`s the selected countries — go-pmtiles treats multiple features as their
each (plus `planet.pmtiles`, always), then `merge`s them if there was union already, no union/dissolve math needed on this side) rather than
more than one — typically seconds, not the tens of minutes an earlier `--bbox=` for precise, per-country extraction. terrain.pmtiles extracts
version of this (fetching every individual Terrarium tile itself) used `planet.pmtiles` (always) plus whichever Mapterhorn regional chunks
to take for a real-sized region. intersect the selection, `merge`s them if there was more than one —
always safe regardless of how many/which countries are selected, since
each chunk is Mapterhorn's own pre-partitioned, non-overlapping source
file (extracting the *same* source once per country and merging those,
by contrast, would fail — adjacent countries' region-extracts share
border tiles, since extraction copies whole tiles rather than clipping
vector content to the polygon, and `pmtiles merge` refuses any
overlapping tile id between inputs).
A build runs in the background (`POST /api/offline-tiles/build` / A build runs in the background (`GET /api/offline-regions` to list/see
`/api/terrain-tiles/build`, polled via the matching `*/build-status` the current selection, `POST` to change it, then `POST
route) — only one of each kind at a time; starting a second while one's /api/offline-tiles/build` / `/api/terrain-tiles/build`, polled via the
running gets a 409. matching `*/build-status` route) — only one of each kind at a time;
starting a second while one's running gets a 409, as does starting
either with nothing selected yet.
== Camper-relevant POIs (Overpass) == Camper-relevant POIs (Overpass)
@@ -0,0 +1 @@
{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"name":"Albania","iso_a2":"AL","iso_a3":"ALB"},"geometry":{"type":"Polygon","coordinates":[[[21.02004,40.842727],[20.99999,40.580004],[20.674997,40.435],[20.615,40.110007],[20.150016,39.624998],[19.98,39.694993],[19.960002,39.915006],[19.406082,40.250773],[19.319059,40.72723],[19.40355,41.409566],[19.540027,41.719986],[19.371769,41.877548],[19.371768,41.877551],[19.304486,42.195745],[19.738051,42.688247],[19.801613,42.500093],[20.0707,42.58863],[20.283755,42.32026],[20.52295,42.21787],[20.590247,41.855409],[20.590247,41.855404],[20.463175,41.515089],[20.605182,41.086226],[21.02004,40.842727]]]}},{"type":"Feature","properties":{"name":"Austria","iso_a2":"AT","iso_a3":"AUT"},"geometry":{"type":"Polygon","coordinates":[[[16.979667,48.123497],[16.903754,47.714866],[16.340584,47.712902],[16.534268,47.496171],[16.202298,46.852386],[16.011664,46.683611],[15.137092,46.658703],[14.632472,46.431817],[13.806475,46.509306],[12.376485,46.767559],[12.153088,47.115393],[11.164828,46.941579],[11.048556,46.751359],[10.442701,46.893546],[9.932448,46.920728],[9.47997,47.10281],[9.632932,47.347601],[9.594226,47.525058],[9.896068,47.580197],[10.402084,47.302488],[10.544504,47.566399],[11.426414,47.523766],[12.141357,47.703083],[12.62076,47.672388],[12.932627,47.467646],[13.025851,47.637584],[12.884103,48.289146],[13.243357,48.416115],[13.595946,48.877172],[14.338898,48.555305],[14.901447,48.964402],[15.253416,49.039074],[16.029647,48.733899],[16.499283,48.785808],[16.960288,48.596982],[16.879983,48.470013],[16.979667,48.123497]]]}},{"type":"Feature","properties":{"name":"Belarus","iso_a2":"BY","iso_a3":"BLR"},"geometry":{"type":"Polygon","coordinates":[[[28.176709,56.16913],[29.229513,55.918344],[29.371572,55.670091],[29.896294,55.789463],[30.873909,55.550976],[30.971836,55.081548],[30.757534,54.811771],[31.384472,54.157056],[31.791424,53.974639],[31.731273,53.794029],[32.405599,53.618045],[32.693643,53.351421],[32.304519,53.132726],[31.49764,53.16743],[31.305201,53.073996],[31.540018,52.742052],[31.78597,52.10168],[31.785992,52.101678],[30.927549,52.042353],[30.619454,51.822806],[30.555117,51.319503],[30.157364,51.416138],[29.254938,51.368234],[28.992835,51.602044],[28.617613,51.427714],[28.241615,51.572227],[27.454066,51.592303],[26.337959,51.832289],[25.327788,51.910656],[24.553106,51.888461],[24.005078,51.617444],[23.527071,51.578454],[23.508002,52.023647],[23.199494,52.486977],[23.799199,52.691099],[23.804935,53.089731],[23.527536,53.470122],[23.484128,53.912498],[24.450684,53.905702],[25.536354,54.282423],[25.768433,54.846963],[26.588279,55.167176],[26.494331,55.615107],[27.10246,55.783314],[28.176709,56.16913]]]}},{"type":"Feature","properties":{"name":"Belgium","iso_a2":"BE","iso_a3":"BEL"},"geometry":{"type":"Polygon","coordinates":[[[6.156658,50.803721],[6.043073,50.128052],[5.782417,50.090328],[5.674052,49.529484],[4.799222,49.985373],[4.286023,49.907497],[3.588184,50.378992],[3.123252,50.780363],[2.658422,50.796848],[2.513573,51.148506],[3.314971,51.345781],[3.315011,51.345777],[3.314971,51.345755],[4.047071,51.267259],[4.973991,51.475024],[5.606976,51.037298],[6.156658,50.803721]]]}},{"type":"Feature","properties":{"name":"Bosnia and Herz.","iso_a2":"BA","iso_a3":"BIH"},"geometry":{"type":"Polygon","coordinates":[[[18.56,42.65],[17.674922,43.028563],[17.297373,43.446341],[16.916156,43.667722],[16.456443,44.04124],[16.23966,44.351143],[15.750026,44.818712],[15.959367,45.233777],[16.318157,45.004127],[16.534939,45.211608],[17.002146,45.233777],[17.861783,45.06774],[18.553214,45.08159],[19.005485,44.860234],[19.00548,44.86023],[19.36803,44.863],[19.11761,44.42307],[19.59976,44.03847],[19.454,43.5681],[19.21852,43.52384],[19.03165,43.43253],[18.70648,43.20011],[18.56,42.65]]]}},{"type":"Feature","properties":{"name":"Bulgaria","iso_a2":"BG","iso_a3":"BGR"},"geometry":{"type":"Polygon","coordinates":[[[22.65715,44.234923],[22.944832,43.823785],[23.332302,43.897011],[24.100679,43.741051],[25.569272,43.688445],[26.065159,43.943494],[27.2424,44.175986],[27.970107,43.812468],[28.558081,43.707462],[28.039095,43.293172],[27.673898,42.577892],[27.99672,42.007359],[27.135739,42.141485],[26.117042,41.826905],[26.106138,41.328899],[25.197201,41.234486],[24.492645,41.583896],[23.692074,41.309081],[22.952377,41.337994],[22.881374,41.999297],[22.380526,42.32026],[22.545012,42.461362],[22.436595,42.580321],[22.604801,42.898519],[22.986019,43.211161],[22.500157,43.642814],[22.410446,44.008063],[22.65715,44.234923]]]}},{"type":"Feature","properties":{"name":"Croatia","iso_a2":"HR","iso_a3":"HRV"},"geometry":{"type":"Polygon","coordinates":[[[16.564808,46.503751],[16.882515,46.380632],[17.630066,45.951769],[18.456062,45.759481],[18.829825,45.908872],[19.072769,45.521511],[19.390476,45.236516],[19.005485,44.860234],[18.553214,45.08159],[17.861783,45.06774],[17.002146,45.233777],[16.534939,45.211608],[16.318157,45.004127],[15.959367,45.233777],[15.750026,44.818712],[Line truncated
+122 -28
View File
@@ -6,17 +6,28 @@ require "tmpdir"
require "fileutils" require "fileutils"
module Maptool module Maptool
# Regenerates data/offline.pmtiles and data/terrain.pmtiles for a given # Regenerates data/offline.pmtiles and data/terrain.pmtiles for the
# bbox by shelling out to go-pmtiles — the same tool docs/development.adoc # currently *selected* regions (data/offline_regions.json, an array of
# already documents running by hand. No job queue: each build runs on its # Natural Earth iso_a2 country codes — see available_regions/
# own background Thread and reports into a Mutex-guarded status Hash a # selected_regions/set_selected_regions below) by shelling out to
# route can poll, the same shape app.rb's own GNSS poller (GNSS_LATEST/ # go-pmtiles — the same tool docs/development.adoc already documents
# GNSS_LATEST_MUTEX) already uses. # running by hand. No job queue: each build runs on its own background
# Thread and reports into a Mutex-guarded status Hash a route can poll,
# the same shape app.rb's own GNSS poller (GNSS_LATEST/GNSS_LATEST_MUTEX)
# already uses.
module PmtilesBuilder module PmtilesBuilder
class BuildError < StandardError; end class BuildError < StandardError; end
# Natural Earth (public domain, no attribution required) 110m admin-0
# countries, filtered to Europe and trimmed to name/iso_a2/iso_a3 —
# see the plan this was built from for the exact filter. 110m is
# deliberately coarse (a km or so of border precision, not
# survey-grade) — fine for "which tiles to grab," a same-schema swap
# to Natural Earth's 50m/10m cut if that ever needs to be tighter.
REGIONS_FILE = File.join(__dir__, "data", "europe_countries.geojson")
# Protomaps' own daily full-planet basemap build index — extract_offline # Protomaps' own daily full-planet basemap build index — extract_offline
# below only pulls the HTTP byte ranges covering the requested bbox # below only pulls the HTTP byte ranges covering the requested region
# (go-pmtiles's `extract` supports a remote source over range requests), # (go-pmtiles's `extract` supports a remote source over range requests),
# never the whole ~120GB file. # never the whole ~120GB file.
PROTOMAPS_BUILDS_INDEX = "https://build-metadata.protomaps.dev/builds.json" PROTOMAPS_BUILDS_INDEX = "https://build-metadata.protomaps.dev/builds.json"
@@ -25,7 +36,7 @@ module Maptool
# Mapterhorn (a Protomaps-affiliated project) publishes Terrarium- # Mapterhorn (a Protomaps-affiliated project) publishes Terrarium-
# encoded terrain as pre-built PMTiles archives: one global # encoded terrain as pre-built PMTiles archives: one global
# planet.pmtiles (z0-12) plus ~450 regional chunks (z13 up to a # planet.pmtiles (z0-12) plus ~450 regional chunks (z13 up to a
# per-region max, 13-18) — same "extract only the bbox's byte ranges # per-region max, 13-18) — same "extract only the region's byte ranges
# from a remote archive" story as offline.pmtiles above, so no more # from a remote archive" story as offline.pmtiles above, so no more
# fetching individual tiles or building an MBTiles file ourselves the # fetching individual tiles or building an MBTiles file ourselves the
# way an earlier version of this file did. # way an earlier version of this file did.
@@ -59,25 +70,97 @@ module Maptool
TERRAIN_MUTEX.synchronize { TERRAIN_STATUS.replace(state: "idle") } TERRAIN_MUTEX.synchronize { TERRAIN_STATUS.replace(state: "idle") }
end end
# bounds: [south, west, north, east]. Returns once the background Thread # --- region selection ---
# has started, not once the build finishes — raises BuildError (the
# caller turns this into a 409) if one's already running. def self.available_regions
region_features.map { |f| { code: f["properties"]["iso_a2"], name: f["properties"]["name"] } }.sort_by { |r| r[:name] }
end
def self.selected_regions(data_dir)
path = selected_regions_path(data_dir)
return [] unless File.exist?(path)
JSON.parse(File.read(path))
rescue JSON::ParserError
[]
end
def self.set_selected_regions(data_dir, codes)
unknown = codes - available_regions.map { |r| r[:code] }
raise BuildError, "unknown region code(s): #{unknown.join(", ")}" unless unknown.empty?
File.write(selected_regions_path(data_dir), JSON.generate(codes))
end
def self.selected_regions_path(data_dir)
File.join(data_dir, "offline_regions.json")
end
private_class_method :selected_regions_path
def self.region_features
@region_features ||= JSON.parse(File.read(REGIONS_FILE))["features"]
end
private_class_method :region_features
# Writes the FeatureCollection of just the selected countries into
# `dir` — this *is* the file go-pmtiles' `--region=` reads. No
# union/dissolve needed: go-pmtiles treats a FeatureCollection's
# features as one combined MultiPolygon on its own (confirmed against
# its own source, pmtiles/region.go's UnmarshalRegion). Also returns a
# plain aggregate [south,west,north,east] (min/max over every ring
# coordinate, no geometry library) used only for the Mapterhorn
# chunk-intersection check below — approximate is fine there, the
# `--region=` file is what does the real precise selection.
def self.region_geojson_and_bbox(codes, dir)
selected = region_features.select { |f| codes.include?(f["properties"]["iso_a2"]) }
raise BuildError, "no matching regions for #{codes.join(", ")}" if selected.empty?
path = File.join(dir, "region.geojson")
File.write(path, JSON.generate(type: "FeatureCollection", features: selected))
coords = selected.flat_map { |f| ring_coordinates(f["geometry"]) }
lons = coords.map { |lng, _lat| lng }
lats = coords.map { |_lng, lat| lat }
[path, [lats.min, lons.min, lats.max, lons.max]]
end
private_class_method :region_geojson_and_bbox
def self.ring_coordinates(geometry)
case geometry["type"]
when "Polygon" then geometry["coordinates"].flatten(1)
when "MultiPolygon" then geometry["coordinates"].flatten(2)
else []
end
end
private_class_method :ring_coordinates
# --- builds ---
# Returns once the background Thread has started, not once the build
# finishes — raises BuildError (the caller turns this into an error
# response) if nothing's selected yet, or one's already running.
# #
# fetch_build_url exists as an injectable seam so tests can exercise the # fetch_build_url exists as an injectable seam so tests can exercise the
# rest of this (temp path, go-pmtiles invocation, atomic rename) without # rest of this (temp path, go-pmtiles invocation, atomic rename) without
# a real network call — same reasoning test_helper.rb already gives for # a real network call — same reasoning test_helper.rb already gives for
# keeping /api/plugins route tests off the real Overpass API. # keeping /api/plugins route tests off the real Overpass API.
def self.build_offline(bounds, data_dir:, fetch_build_url: method(:latest_protomaps_build_url)) def self.build_offline(data_dir:, fetch_build_url: method(:latest_protomaps_build_url))
codes = selected_regions(data_dir)
raise BuildError, "select at least one region first" if codes.empty?
begin_run!(OFFLINE_MUTEX, OFFLINE_STATUS) begin_run!(OFFLINE_MUTEX, OFFLINE_STATUS)
Thread.new { run_job(OFFLINE_MUTEX, OFFLINE_STATUS) { extract_offline(bounds, data_dir, fetch_build_url) } } Thread.new { run_job(OFFLINE_MUTEX, OFFLINE_STATUS) { extract_offline(codes, data_dir, fetch_build_url) } }
end end
# fetch_manifest: same injectable-seam reasoning as fetch_build_url # fetch_manifest: same injectable-seam reasoning as fetch_build_url
# above, for the one Mapterhorn manifest lookup this needs instead of # above, for the one Mapterhorn manifest lookup this needs instead of
# the Protomaps build index. # the Protomaps build index.
def self.build_terrain(bounds, data_dir:, fetch_manifest: method(:fetch_mapterhorn_manifest)) def self.build_terrain(data_dir:, fetch_manifest: method(:fetch_mapterhorn_manifest))
codes = selected_regions(data_dir)
raise BuildError, "select at least one region first" if codes.empty?
begin_run!(TERRAIN_MUTEX, TERRAIN_STATUS) begin_run!(TERRAIN_MUTEX, TERRAIN_STATUS)
Thread.new { run_job(TERRAIN_MUTEX, TERRAIN_STATUS) { build_terrain_pmtiles(bounds, data_dir, fetch_manifest) } } Thread.new { run_job(TERRAIN_MUTEX, TERRAIN_STATUS) { build_terrain_pmtiles(codes, data_dir, fetch_manifest) } }
end end
def self.begin_run!(mutex, status) def self.begin_run!(mutex, status)
@@ -98,14 +181,16 @@ module Maptool
# --- offline.pmtiles --- # --- offline.pmtiles ---
def self.extract_offline(bounds, data_dir, fetch_build_url) def self.extract_offline(codes, data_dir, fetch_build_url)
source = fetch_build_url.call source = fetch_build_url.call
south, west, north, east = bounds scratch_dir = Dir.mktmpdir("maptool-offline")
region_path, = region_geojson_and_bbox(codes, scratch_dir)
tmp_path = File.join(data_dir, "offline.pmtiles.tmp") tmp_path = File.join(data_dir, "offline.pmtiles.tmp")
run_pmtiles!("extract", source, tmp_path, "--bbox=#{west},#{south},#{east},#{north}", "--maxzoom=15") run_pmtiles!("extract", source, tmp_path, "--region=#{region_path}", "--maxzoom=15")
File.rename(tmp_path, File.join(data_dir, "offline.pmtiles")) File.rename(tmp_path, File.join(data_dir, "offline.pmtiles"))
ensure ensure
FileUtils.remove_entry(scratch_dir) if scratch_dir && Dir.exist?(scratch_dir)
File.delete(tmp_path) if tmp_path && File.exist?(tmp_path) File.delete(tmp_path) if tmp_path && File.exist?(tmp_path)
end end
private_class_method :extract_offline private_class_method :extract_offline
@@ -126,25 +211,34 @@ module Maptool
# --- terrain.pmtiles --- # --- terrain.pmtiles ---
# planet.pmtiles (z0-12, always) plus whichever regional chunks # planet.pmtiles (z0-12, always) plus whichever regional chunks
# intersect the bbox (z13+, zero or more — most bboxes hit exactly one, # intersect the selection's aggregate bbox (z13+, zero or more) are
# a bbox straddling a chunk boundary hits two) are each `extract`ed on # each `extract`ed on their own using the *same* --region= file, then
# their own, then combined with `pmtiles merge` if there was more than # combined with `pmtiles merge` if there was more than one. Always
# one (planet + a chunk are disjoint by zoom level, so this is always a # safe to merge regardless of how many countries are selected or
# valid merge; go-pmtiles requires clustered, matching-tile-type inputs, # whether they're adjacent: each Mapterhorn chunk is its own
# which every Mapterhorn PMTiles archive already satisfies). # pre-partitioned, non-overlapping source grid cell, so two different
def self.build_terrain_pmtiles(bounds, data_dir, fetch_manifest) # chunks' extracts can never share a tile id — the only thing `merge`
south, west, north, east = bounds # refuses (see pmtiles/merge.go's own disjointness check). What would
# NOT be safe is extracting the *same* source once per country and
# merging those — adjacent countries' region-extracts would share
# border tiles (extraction copies whole tiles, it doesn't clip vector
# content to the polygon) — so this always does one extract per
# *source archive*, never one per country.
def self.build_terrain_pmtiles(codes, data_dir, fetch_manifest)
items = fetch_manifest.call items = fetch_manifest.call
planet = items.find { |item| item["name"] == MAPTERHORN_PLANET_NAME } planet = items.find { |item| item["name"] == MAPTERHORN_PLANET_NAME }
raise BuildError, "mapterhorn manifest is missing #{MAPTERHORN_PLANET_NAME}" unless planet raise BuildError, "mapterhorn manifest is missing #{MAPTERHORN_PLANET_NAME}" unless planet
scratch_dir = Dir.mktmpdir("maptool-terrain")
region_path, bbox = region_geojson_and_bbox(codes, scratch_dir)
south, west, north, east = bbox
chunks = items.select { |item| item["name"] != MAPTERHORN_PLANET_NAME && bbox_intersects?(item, south, west, north, east) } chunks = items.select { |item| item["name"] != MAPTERHORN_PLANET_NAME && bbox_intersects?(item, south, west, north, east) }
scratch_dir = Dir.mktmpdir("maptool-terrain")
tmp_path = File.join(data_dir, "terrain.pmtiles.tmp") tmp_path = File.join(data_dir, "terrain.pmtiles.tmp")
extracts = [planet, *chunks].each_with_index.map do |item, i| extracts = [planet, *chunks].each_with_index.map do |item, i|
extract_path = File.join(scratch_dir, "extract-#{i}.pmtiles") extract_path = File.join(scratch_dir, "extract-#{i}.pmtiles")
run_pmtiles!("extract", item["url"], extract_path, "--bbox=#{west},#{south},#{east},#{north}", "--maxzoom=#{TERRAIN_MAX_ZOOM}") run_pmtiles!("extract", item["url"], extract_path, "--region=#{region_path}", "--maxzoom=#{TERRAIN_MAX_ZOOM}")
extract_path extract_path
end end
+38 -12
View File
@@ -401,39 +401,65 @@ html, body {
padding: 8px 12px; padding: 8px 12px;
} }
.offline-data-panel__row + .offline-data-panel__row { .offline-data-panel__regions-label {
margin-top: 10px; margin: 0 0 4px;
padding-top: 10px; font-weight: 600;
border-top: 1px solid #eee;
} }
.offline-data-panel__row-header { .offline-data-panel__region-list {
list-style: none;
margin: 0;
padding: 0;
max-height: 220px;
overflow-y: auto;
border: 1px solid #eee;
border-radius: 4px;
}
.offline-data-panel__region-list label {
display: flex; display: flex;
align-items: center; align-items: center;
justify-content: space-between; gap: 6px;
gap: 8px; padding: 4px 8px;
cursor: pointer;
} }
.offline-data-panel__build { .offline-data-panel__region-list label:hover {
flex-shrink: 0; background: #f0f0f0;
}
.offline-data-panel__region-status {
padding: 6px 8px;
color: #888;
}
.offline-data-panel__apply {
margin-top: 8px;
width: 100%;
border: 1px solid #ccc; border: 1px solid #ccc;
border-radius: 4px; border-radius: 4px;
background: #fff; background: #fff;
padding: 4px 8px; padding: 6px 8px;
font: inherit; font: inherit;
font-size: 12px; font-size: 12px;
cursor: pointer; cursor: pointer;
} }
.offline-data-panel__build:hover { .offline-data-panel__apply:hover {
background: #f0f0f0; background: #f0f0f0;
} }
.offline-data-panel__build:disabled { .offline-data-panel__apply:disabled {
opacity: 0.5; opacity: 0.5;
cursor: default; cursor: default;
} }
.offline-data-panel__row {
margin-top: 14px;
padding-top: 10px;
border-top: 1px solid #eee;
}
.offline-data-panel__status { .offline-data-panel__status {
margin: 6px 0 0; margin: 6px 0 0;
color: #666; color: #666;
+1 -1
View File
@@ -516,7 +516,7 @@ if (isLoggedIn()) new PluginsControl(map).mount(sidebar);
sidebar.insertAdjacentHTML("beforeend", `<div class="sidebar-group-label" title="Offline hub"></div>`); sidebar.insertAdjacentHTML("beforeend", `<div class="sidebar-group-label" title="Offline hub"></div>`);
layerToggle.mount(sidebar); layerToggle.mount(sidebar);
layerToggle.setActive("online"); layerToggle.setActive("online");
new OfflineDataControl(map).mount(sidebar); new OfflineDataControl().mount(sidebar);
sidebar.insertAdjacentHTML("beforeend", `<div class="sidebar-group-label" title="Navigation"></div>`); sidebar.insertAdjacentHTML("beforeend", `<div class="sidebar-group-label" title="Navigation"></div>`);
gnssControl.mount(sidebar); gnssControl.mount(sidebar);
+97 -52
View File
@@ -1,6 +1,7 @@
import { OFFLINE_DATA_ICON } from "/js/icons.js"; import { OFFLINE_DATA_ICON } from "/js/icons.js";
import { notifySidePanelOpen, onlyOneSidePanelOpen } from "/js/panels.js"; import { notifySidePanelOpen, onlyOneSidePanelOpen } from "/js/panels.js";
import { isLoggedIn } from "/js/auth.js"; import { isLoggedIn } from "/js/auth.js";
import { escapeHtml } from "/js/gpx.js";
const POLL_INTERVAL_MS = 3000; const POLL_INTERVAL_MS = 3000;
@@ -16,51 +17,106 @@ function formatElapsed(ms) {
return minutes > 0 ? `${minutes}m ${seconds}s` : `${seconds}s`; return minutes > 0 ? `${minutes}m ${seconds}s` : `${seconds}s`;
} }
// "Building…" alone gave no way to tell a slow build (terrain.pmtiles can // "Building…" alone gave no way to tell a slow build from a stuck one —
// take minutes — lib/maptool/pmtiles_builder.rb fetches tiles one at a // elapsed time answers "is it alive" regardless of how long a given
// time) from a stuck one. Elapsed time alone answers "is it alive" for // selection takes.
// both files; tile counts (only terrain.pmtiles reports these) give an
// actual sense of how much is left.
function formatRunning(status) { function formatRunning(status) {
const elapsed = status.started_at ? ` (${formatElapsed(Date.now() - new Date(status.started_at).getTime())})` : ""; const elapsed = status.started_at ? ` (${formatElapsed(Date.now() - new Date(status.started_at).getTime())})` : "";
const tiles = status.tiles_total ? ` · ${status.tiles_done}/${status.tiles_total} tiles` : ""; return `Building…${elapsed}`;
return `Building…${elapsed}${tiles}`;
} }
// One row of the Offline data panel — status text plus a "Regenerate for // The checklist of selectable countries (GET /api/offline-regions) plus
// current view" button, for either offline.pmtiles or terrain.pmtiles. // the "Apply changes" button that saves the ticked set (POST, same URL)
// Both files behave identically from here (same status/build/build-status // and hands it off — doesn't trigger a build itself, OfflineDataControl
// endpoint shape, see lib/maptool/pmtiles_builder.rb), just against // does that across both files once the selection is saved.
// different URLs/labels, so one class covers both rather than writing the class RegionPicker {
// same fetch/poll logic twice. constructor(onApply) {
this._onApply = onApply;
}
async mount(container) {
const section = document.createElement("div");
section.className = "offline-data-panel__regions";
section.innerHTML = `
<p class="offline-data-panel__regions-label">Regions</p>
<ul class="offline-data-panel__region-list"><li class="offline-data-panel__region-status">Loading…</li></ul>
<button type="button" class="offline-data-panel__apply" hidden>Apply changes</button>
`;
this._list = section.querySelector(".offline-data-panel__region-list");
this._applyButton = section.querySelector(".offline-data-panel__apply");
// Viewing/ticking works for anyone; only saving needs a login — same
// finer-grained gating measure.js/gpx.js use for their own
// write-only affordances, rather than hiding this whole control.
this._applyButton.hidden = !isLoggedIn();
this._applyButton.addEventListener("click", () => this._apply());
container.appendChild(section);
let data;
try {
data = await (await fetch("/api/offline-regions")).json();
} catch {
this._list.innerHTML = `<li class="offline-data-panel__region-status">Could not load the region list.</li>`;
return;
}
const selected = new Set(data.selected);
this._list.innerHTML = data.available
.map(
(region) => `
<li>
<label>
<input type="checkbox" value="${escapeHtml(region.code)}" ${selected.has(region.code) ? "checked" : ""}>
${escapeHtml(region.name)}
</label>
</li>`,
)
.join("");
}
async _apply() {
const codes = [...this._list.querySelectorAll("input:checked")].map((el) => el.value);
const body = new URLSearchParams();
codes.forEach((code) => body.append("codes[]", code));
this._applyButton.disabled = true;
try {
const response = await fetch("/api/offline-regions", { method: "POST", body });
const data = await response.json().catch(() => ({}));
if (!response.ok) throw new Error(data.error || `could not save the selection: ${response.status}`);
} catch (err) {
window.alert(err.message);
return;
} finally {
this._applyButton.disabled = false;
}
this._onApply();
}
}
// One row of the Offline data panel — just a status readout for either
// offline.pmtiles or terrain.pmtiles, both driven by the same selection
// (RegionPicker above). Both files behave identically from here (same
// status/build/build-status endpoint shape, see
// lib/maptool/pmtiles_builder.rb), just against different URLs/labels, so
// one class covers both rather than writing the same fetch/poll logic
// twice.
class DataFileRow { class DataFileRow {
constructor(map, { label, statusUrl, buildUrl, buildStatusUrl }) { constructor({ label, statusUrl, buildUrl, buildStatusUrl }) {
this._map = map;
this._label = label; this._label = label;
this._statusUrl = statusUrl; this._statusUrl = statusUrl;
this._buildUrl = buildUrl; this._buildUrl = buildUrl;
this._buildStatusUrl = buildStatusUrl; this._buildStatusUrl = buildStatusUrl;
this._exists = false;
} }
mount(container) { mount(container) {
const row = document.createElement("div"); const row = document.createElement("div");
row.className = "offline-data-panel__row"; row.className = "offline-data-panel__row";
row.innerHTML = ` row.innerHTML = `
<div class="offline-data-panel__row-header">
<strong>${this._label}</strong> <strong>${this._label}</strong>
<button type="button" class="offline-data-panel__build">Regenerate for current view</button>
</div>
<p class="offline-data-panel__status">Checking…</p> <p class="offline-data-panel__status">Checking…</p>
`; `;
this._buildButton = row.querySelector(".offline-data-panel__build");
this._statusEl = row.querySelector(".offline-data-panel__status"); this._statusEl = row.querySelector(".offline-data-panel__status");
// Viewing status works for anyone; only triggering a build needs a
// login — same finer-grained gating measure.js/gpx.js use for their
// own write-only affordances, rather than hiding this whole control.
this._buildButton.hidden = !isLoggedIn();
this._buildButton.addEventListener("click", () => this._build());
container.appendChild(row); container.appendChild(row);
this._refreshStatus(); this._refreshStatus();
} }
@@ -73,34 +129,21 @@ class DataFileRow {
this._statusEl.textContent = "Could not check status."; this._statusEl.textContent = "Could not check status.";
return; return;
} }
this._exists = data.available;
this._statusEl.textContent = data.available this._statusEl.textContent = data.available
? `${formatBytes(data.size)} · updated ${new Date(data.mtime).toLocaleString([], { dateStyle: "short", timeStyle: "short" })}` ? `${formatBytes(data.size)} · updated ${new Date(data.mtime).toLocaleString([], { dateStyle: "short", timeStyle: "short" })}`
: "Not generated yet."; : "Not generated yet.";
} }
async _build() { // Called by OfflineDataControl once RegionPicker's Apply has saved a
if (this._exists && !window.confirm(`Replace the existing ${this._label.toLowerCase()} with a fresh build for the current view?`)) { // new selection — not tied to a button of this row's own.
return; async triggerBuild() {
}
const bounds = this._map.getBounds();
const body = new URLSearchParams({
south: bounds.getSouth(),
west: bounds.getWest(),
north: bounds.getNorth(),
east: bounds.getEast(),
});
this._buildButton.disabled = true;
this._statusEl.textContent = "Starting…"; this._statusEl.textContent = "Starting…";
try { try {
const response = await fetch(this._buildUrl, { method: "POST", body }); const response = await fetch(this._buildUrl, { method: "POST" });
const data = await response.json().catch(() => ({})); const data = await response.json().catch(() => ({}));
if (!response.ok) throw new Error(data.error || `build failed: ${response.status}`); if (!response.ok) throw new Error(data.error || `build failed: ${response.status}`);
} catch (err) { } catch (err) {
this._statusEl.textContent = err.message; this._statusEl.textContent = err.message;
this._buildButton.disabled = false;
return; return;
} }
@@ -126,7 +169,6 @@ class DataFileRow {
return; return;
} }
this._buildButton.disabled = false;
if (status.state === "error") { if (status.state === "error") {
this._statusEl.textContent = `Build failed: ${status.message}`; this._statusEl.textContent = `Build failed: ${status.message}`;
} else { } else {
@@ -137,27 +179,29 @@ class DataFileRow {
} }
// A sidebar panel wrapping the manual go-pmtiles workflow // A sidebar panel wrapping the manual go-pmtiles workflow
// docs/development.adoc documents — "Regenerate for current view" on // docs/development.adoc documents — a region picker (which countries
// either file shells out to it server-side instead of an admin running it // should be available offline) plus, on "Apply changes", a rebuild of
// by hand (see lib/maptool/pmtiles_builder.rb). No area-picker: the // offline.pmtiles/terrain.pmtiles covering the whole current selection
// current map viewport is the region, same convention the Overpass POI // (see lib/maptool/pmtiles_builder.rb — there's no standalone per-country
// plugin already uses for "give me a bbox." // file to individually delete; "remove a country" and "refresh a
// country" are both just "change the selection, then Apply").
export class OfflineDataControl { export class OfflineDataControl {
constructor(map) { constructor() {
this._rows = [ this._rows = [
new DataFileRow(map, { new DataFileRow({
label: "Offline map data", label: "Offline map data",
statusUrl: "/api/offline-tiles", statusUrl: "/api/offline-tiles",
buildUrl: "/api/offline-tiles/build", buildUrl: "/api/offline-tiles/build",
buildStatusUrl: "/api/offline-tiles/build-status", buildStatusUrl: "/api/offline-tiles/build-status",
}), }),
new DataFileRow(map, { new DataFileRow({
label: "Offline terrain data", label: "Offline terrain data",
statusUrl: "/api/terrain-tiles", statusUrl: "/api/terrain-tiles",
buildUrl: "/api/terrain-tiles/build", buildUrl: "/api/terrain-tiles/build",
buildStatusUrl: "/api/terrain-tiles/build-status", buildStatusUrl: "/api/terrain-tiles/build-status",
}), }),
]; ];
this._regionPicker = new RegionPicker(() => this._rows.forEach((row) => row.triggerBuild()));
} }
mount(sidebar) { mount(sidebar) {
@@ -171,6 +215,7 @@ export class OfflineDataControl {
this._panel.querySelector(".basemap-panel__close").addEventListener("click", () => this._togglePanel(false)); this._panel.querySelector(".basemap-panel__close").addEventListener("click", () => this._togglePanel(false));
const body = this._panel.querySelector(".offline-data-panel__body"); const body = this._panel.querySelector(".offline-data-panel__body");
this._regionPicker.mount(body);
this._rows.forEach((row) => row.mount(body)); this._rows.forEach((row) => row.mount(body));
onlyOneSidePanelOpen(this._panel, () => this._icon.classList.remove("active")); onlyOneSidePanelOpen(this._panel, () => this._icon.classList.remove("active"));
+34 -12
View File
@@ -349,41 +349,63 @@ class AppTest < Minitest::Test
assert_equal 502, response.status assert_equal 502, response.status
end end
# --- offline regions ---
def test_offline_regions_lists_available_and_selected
body = json_body(request.get("/api/offline-regions"))
assert_operator body["available"].size, :>, 0
assert_includes body["available"], { "code" => "SK", "name" => "Slovakia" }
assert_equal [], body["selected"]
end
def test_offline_regions_post_requires_login
assert_equal 401, request.post("/api/offline-regions").status
end
def test_offline_regions_post_saves_the_selection
response = request.post("/api/offline-regions", params: { "codes" => %w[SK AT] }, **logged_in_env)
assert_equal 200, response.status
assert_equal %w[SK AT], json_body(request.get("/api/offline-regions"))["selected"]
end
def test_offline_regions_post_rejects_an_unknown_code
response = request.post("/api/offline-regions", params: { "codes" => ["XX"] }, **logged_in_env)
assert_equal 400, response.status
end
# --- pmtiles regeneration --- # --- pmtiles regeneration ---
# #
# The full build (network + go-pmtiles) is covered in isolation, with # The full build (network + go-pmtiles) is covered in isolation, with
# fakes injected, by test_pmtiles_builder.rb — same "keep this offline" # fakes injected, by test_pmtiles_builder.rb — same "keep this offline"
# reasoning as the rest of this file. What's route-specific and worth # reasoning as the rest of this file. What's route-specific and worth
# covering here is the bbox validation and the 409-while-running guard, # covering here is the empty-selection validation and the
# neither of which needs a real build to actually run. # 409-while-running guard, neither of which needs a real build to
# actually run.
def test_offline_tiles_build_status_defaults_to_idle def test_offline_tiles_build_status_defaults_to_idle
assert_equal({ "state" => "idle" }, json_body(request.get("/api/offline-tiles/build-status"))) assert_equal({ "state" => "idle" }, json_body(request.get("/api/offline-tiles/build-status")))
assert_equal({ "state" => "idle" }, json_body(request.get("/api/terrain-tiles/build-status"))) assert_equal({ "state" => "idle" }, json_body(request.get("/api/terrain-tiles/build-status")))
end end
def test_offline_tiles_build_requires_a_bbox def test_offline_tiles_build_is_409_with_no_selection
response = request.post("/api/offline-tiles/build", **logged_in_env) response = request.post("/api/offline-tiles/build", **logged_in_env)
assert_equal 400, response.status assert_equal 409, response.status
end end
def test_terrain_tiles_build_requires_a_bbox def test_terrain_tiles_build_is_409_with_no_selection
response = request.post("/api/terrain-tiles/build", **logged_in_env) response = request.post("/api/terrain-tiles/build", **logged_in_env)
assert_equal 400, response.status assert_equal 409, response.status
end end
# fetch_build_url blocks on a Queue, called directly (not through the # fetch_build_url blocks on a Queue, called directly (not through the
# route, which has no way to inject a fake) so the build is # route, which has no way to inject a fake) so the build is
# deterministically still "running" when the route call below happens. # deterministically still "running" when the route call below happens.
def test_offline_tiles_build_is_409_when_one_is_already_running def test_offline_tiles_build_is_409_when_one_is_already_running
Maptool::PmtilesBuilder.set_selected_regions(Maptool::App::DATA_DIR, ["SK"])
gate = Queue.new gate = Queue.new
thread = Maptool::PmtilesBuilder.build_offline([1, 2, 3, 4], data_dir: Maptool::App::DATA_DIR, fetch_build_url: -> { gate.pop }) thread = Maptool::PmtilesBuilder.build_offline(data_dir: Maptool::App::DATA_DIR, fetch_build_url: -> { gate.pop })
response = request.post( response = request.post("/api/offline-tiles/build", **logged_in_env)
"/api/offline-tiles/build",
params: { south: "1", west: "2", north: "3", east: "4" },
**logged_in_env,
)
assert_equal 409, response.status assert_equal 409, response.status
ensure ensure
gate.push("unused") gate.push("unused")
+1
View File
@@ -66,6 +66,7 @@ module AppTestHelper
def teardown def teardown
FileUtils.rm_rf(Dir.glob(File.join(Maptool::App::DATA_DIR, "gpx", "*"))) FileUtils.rm_rf(Dir.glob(File.join(Maptool::App::DATA_DIR, "gpx", "*")))
File.write(File.join(Maptool::App::DATA_DIR, "livetrack.json"), "[]") File.write(File.join(Maptool::App::DATA_DIR, "livetrack.json"), "[]")
FileUtils.rm_f(File.join(Maptool::App::DATA_DIR, "offline_regions.json"))
Maptool::PmtilesBuilder.reset! Maptool::PmtilesBuilder.reset!
super super
end end
+80 -36
View File
@@ -11,13 +11,19 @@ require "fileutils"
# CLI (PMTILES_BIN) stands in for the real go-pmtiles binary, logging each # CLI (PMTILES_BIN) stands in for the real go-pmtiles binary, logging each
# invocation's subcommand + first arg to PMTILES_LOG so tests can assert on # invocation's subcommand + first arg to PMTILES_LOG so tests can assert on
# what actually got called (e.g. "was merge invoked, and with what"). # what actually got called (e.g. "was merge invoked, and with what").
#
# Country codes below are real ones from the vendored
# lib/maptool/data/europe_countries.geojson (Slovakia, Iceland, Spain) —
# chosen so their real bboxes fall cleanly inside/outside the fake
# manifest's chunk extents below, without needing a second fake geometry
# source.
class PmtilesBuilderTest < Minitest::Test class PmtilesBuilderTest < Minitest::Test
def setup def setup
@data_dir = Dir.mktmpdir("maptool-pmtiles-test-data") @data_dir = Dir.mktmpdir("maptool-pmtiles-test-data")
@bin_dir = Dir.mktmpdir("maptool-pmtiles-test-bin") @bin_dir = Dir.mktmpdir("maptool-pmtiles-test-bin")
@log = File.join(@bin_dir, "pmtiles.log") @log = File.join(@bin_dir, "pmtiles.log")
# Output path: $3 for `extract SOURCE OUTPUT --bbox=... --maxzoom=...` # Output path: $3 for `extract SOURCE OUTPUT --region=... --maxzoom=...`
# and `convert INPUT OUTPUT` alike; the *last* arg for `merge INPUT_1 # and `convert INPUT OUTPUT` alike; the *last* arg for `merge INPUT_1
# [INPUT_2 ...] OUTPUT`, since its input count varies. # [INPUT_2 ...] OUTPUT`, since its input count varies.
@pmtiles_ok = File.join(@bin_dir, "pmtiles-ok") @pmtiles_ok = File.join(@bin_dir, "pmtiles-ok")
@@ -57,15 +63,17 @@ class PmtilesBuilderTest < Minitest::Test
File.exist?(@log) ? File.readlines(@log, chomp: true) : [] File.exist?(@log) ? File.readlines(@log, chomp: true) : []
end end
# planet.pmtiles always covers the whole world (z0-12); chunk-a/chunk-b def select_regions(*codes)
# are disjoint small regions (z13+), matching Mapterhorn's real shape Maptool::PmtilesBuilder.set_selected_regions(@data_dir, codes)
# (see download_urls.json) closely enough to exercise bbox_intersects? end
# and the extract-then-maybe-merge logic without a real manifest fetch.
# chunk-a covers Slovakia's real extent, chunk-b covers Iceland's —
# Spain falls in neither, for the "no chunk intersects" case.
def fake_manifest def fake_manifest
[ [
{ "name" => "planet.pmtiles", "url" => "https://example.com/planet.pmtiles", "min_lon" => -180.0, "min_lat" => -85.0, "max_lon" => 180.0, "max_lat" => 85.0, "min_zoom" => 0, "max_zoom" => 12 }, { "name" => "planet.pmtiles", "url" => "https://example.com/planet.pmtiles", "min_lon" => -180.0, "min_lat" => -85.0, "max_lon" => 180.0, "max_lat" => 85.0, "min_zoom" => 0, "max_zoom" => 12 },
{ "name" => "chunk-a.pmtiles", "url" => "https://example.com/chunk-a.pmtiles", "min_lon" => 17.0, "min_lat" => 48.0, "max_lon" => 17.5, "max_lat" => 48.5, "min_zoom" => 13, "max_zoom" => 15 }, { "name" => "chunk-a.pmtiles", "url" => "https://example.com/chunk-a.pmtiles", "min_lon" => 16.0, "min_lat" => 47.0, "max_lon" => 23.0, "max_lat" => 50.0, "min_zoom" => 13, "max_zoom" => 15 },
{ "name" => "chunk-b.pmtiles", "url" => "https://example.com/chunk-b.pmtiles", "min_lon" => 20.0, "min_lat" => 50.0, "max_lon" => 20.5, "max_lat" => 50.5, "min_zoom" => 13, "max_zoom" => 15 }, { "name" => "chunk-b.pmtiles", "url" => "https://example.com/chunk-b.pmtiles", "min_lon" => -25.0, "min_lat" => 63.0, "max_lon" => -13.0, "max_lat" => 67.0, "min_zoom" => 13, "max_zoom" => 15 },
] ]
end end
@@ -74,27 +82,51 @@ class PmtilesBuilderTest < Minitest::Test
assert_equal({ state: "idle" }, Maptool::PmtilesBuilder.terrain_status) assert_equal({ state: "idle" }, Maptool::PmtilesBuilder.terrain_status)
end end
# --- region selection ---
def test_available_regions_includes_known_countries
codes = Maptool::PmtilesBuilder.available_regions.map { |r| r[:code] }
assert_includes codes, "SK"
assert_includes codes, "IS"
end
def test_selected_regions_defaults_to_empty
assert_empty Maptool::PmtilesBuilder.selected_regions(@data_dir)
end
def test_set_selected_regions_round_trips
select_regions("SK", "AT")
assert_equal ["SK", "AT"], Maptool::PmtilesBuilder.selected_regions(@data_dir)
end
def test_set_selected_regions_rejects_an_unknown_code
assert_raises(Maptool::PmtilesBuilder::BuildError) { select_regions("SK", "XX") }
end
# --- offline.pmtiles ---
def test_build_offline_raises_with_no_selection
assert_raises(Maptool::PmtilesBuilder::BuildError) do
Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "unused" })
end
end
def test_build_offline_writes_the_file_and_reports_done def test_build_offline_writes_the_file_and_reports_done
select_regions("SK")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
thread = Maptool::PmtilesBuilder.build_offline( thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "https://example.com/fake.pmtiles" })
[1, 2, 3, 4],
data_dir: @data_dir,
fetch_build_url: -> { "https://example.com/fake.pmtiles" },
)
thread.join thread.join
assert_path_exists File.join(@data_dir, "offline.pmtiles") assert_path_exists File.join(@data_dir, "offline.pmtiles")
refute_path_exists File.join(@data_dir, "offline.pmtiles.tmp") refute_path_exists File.join(@data_dir, "offline.pmtiles.tmp")
assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state] assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state]
assert_equal ["extract https://example.com/fake.pmtiles"], log_lines
end end
def test_build_offline_reports_an_error_status_on_non_zero_exit def test_build_offline_reports_an_error_status_on_non_zero_exit
select_regions("SK")
ENV["PMTILES_BIN"] = @pmtiles_fail ENV["PMTILES_BIN"] = @pmtiles_fail
thread = Maptool::PmtilesBuilder.build_offline( thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "https://example.com/fake.pmtiles" })
[1, 2, 3, 4],
data_dir: @data_dir,
fetch_build_url: -> { "https://example.com/fake.pmtiles" },
)
thread.join thread.join
refute_path_exists File.join(@data_dir, "offline.pmtiles") refute_path_exists File.join(@data_dir, "offline.pmtiles")
@@ -104,12 +136,9 @@ class PmtilesBuilderTest < Minitest::Test
end end
def test_build_offline_reports_a_clear_error_when_the_binary_is_missing def test_build_offline_reports_a_clear_error_when_the_binary_is_missing
select_regions("SK")
ENV["PMTILES_BIN"] = File.join(@bin_dir, "does-not-exist") ENV["PMTILES_BIN"] = File.join(@bin_dir, "does-not-exist")
thread = Maptool::PmtilesBuilder.build_offline( thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "https://example.com/fake.pmtiles" })
[1, 2, 3, 4],
data_dir: @data_dir,
fetch_build_url: -> { "https://example.com/fake.pmtiles" },
)
thread.join thread.join
status = Maptool::PmtilesBuilder.offline_status status = Maptool::PmtilesBuilder.offline_status
@@ -121,23 +150,33 @@ class PmtilesBuilderTest < Minitest::Test
# still "running" when the second call happens, rather than racing a real # still "running" when the second call happens, rather than racing a real
# one to see which state wins. # one to see which state wins.
def test_build_offline_raises_when_one_is_already_running def test_build_offline_raises_when_one_is_already_running
select_regions("SK")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
gate = Queue.new gate = Queue.new
thread = Maptool::PmtilesBuilder.build_offline([1, 2, 3, 4], data_dir: @data_dir, fetch_build_url: -> { gate.pop }) thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { gate.pop })
assert_raises(Maptool::PmtilesBuilder::BuildError) do assert_raises(Maptool::PmtilesBuilder::BuildError) do
Maptool::PmtilesBuilder.build_offline([1, 2, 3, 4], data_dir: @data_dir, fetch_build_url: -> { "unused" }) Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, fetch_build_url: -> { "unused" })
end end
ensure ensure
gate.push("https://example.com/fake.pmtiles") gate.push("https://example.com/fake.pmtiles")
thread&.join thread&.join
end end
# A bbox nowhere near chunk-a/chunk-b — only planet.pmtiles matches, so # --- terrain.pmtiles ---
def test_build_terrain_raises_with_no_selection
assert_raises(Maptool::PmtilesBuilder::BuildError) do
Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
end
end
# Spain intersects neither fake chunk — only planet.pmtiles matches, so
# this should be a single extract straight to the final path, no merge. # this should be a single extract straight to the final path, no merge.
def test_build_terrain_extracts_only_planet_when_no_chunk_intersects def test_build_terrain_extracts_only_planet_when_no_chunk_intersects
select_regions("ES")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { fake_manifest }) thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
thread.join thread.join
assert_path_exists File.join(@data_dir, "terrain.pmtiles") assert_path_exists File.join(@data_dir, "terrain.pmtiles")
@@ -145,11 +184,12 @@ class PmtilesBuilderTest < Minitest::Test
assert_equal ["extract https://example.com/planet.pmtiles"], log_lines assert_equal ["extract https://example.com/planet.pmtiles"], log_lines
end end
# A bbox inside chunk-a's extent — planet.pmtiles (always) plus that one # Slovakia intersects chunk-a only — planet.pmtiles (always) plus that
# chunk get extracted, then merged. # one chunk get extracted, then merged.
def test_build_terrain_extracts_and_merges_a_single_intersecting_chunk def test_build_terrain_extracts_and_merges_a_single_intersecting_chunk
select_regions("SK")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
thread = Maptool::PmtilesBuilder.build_terrain([48.1, 17.1, 48.2, 17.2], data_dir: @data_dir, fetch_manifest: -> { fake_manifest }) thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
thread.join thread.join
assert_path_exists File.join(@data_dir, "terrain.pmtiles") assert_path_exists File.join(@data_dir, "terrain.pmtiles")
@@ -160,11 +200,12 @@ class PmtilesBuilderTest < Minitest::Test
assert_equal 1, merged.size assert_equal 1, merged.size
end end
# A bbox spanning both chunk-a and chunk-b — planet + both chunks, one # Slovakia + Iceland together intersect both fake chunks — planet + both
# 3-way merge. # chunks, one 3-way merge.
def test_build_terrain_merges_multiple_intersecting_chunks def test_build_terrain_merges_multiple_intersecting_chunks
select_regions("SK", "IS")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
thread = Maptool::PmtilesBuilder.build_terrain([47.9, 16.9, 50.6, 20.6], data_dir: @data_dir, fetch_manifest: -> { fake_manifest }) thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
thread.join thread.join
assert_equal "done", Maptool::PmtilesBuilder.terrain_status[:state] assert_equal "done", Maptool::PmtilesBuilder.terrain_status[:state]
@@ -174,8 +215,9 @@ class PmtilesBuilderTest < Minitest::Test
end end
def test_build_terrain_reports_a_clear_error_when_the_manifest_is_missing_planet def test_build_terrain_reports_a_clear_error_when_the_manifest_is_missing_planet
select_regions("ES")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { fake_manifest.reject { |i| i["name"] == "planet.pmtiles" } }) thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest.reject { |i| i["name"] == "planet.pmtiles" } })
thread.join thread.join
status = Maptool::PmtilesBuilder.terrain_status status = Maptool::PmtilesBuilder.terrain_status
@@ -184,8 +226,9 @@ class PmtilesBuilderTest < Minitest::Test
end end
def test_build_terrain_reports_an_error_status_on_non_zero_exit def test_build_terrain_reports_an_error_status_on_non_zero_exit
select_regions("ES")
ENV["PMTILES_BIN"] = @pmtiles_fail ENV["PMTILES_BIN"] = @pmtiles_fail
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { fake_manifest }) thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { fake_manifest })
thread.join thread.join
refute_path_exists File.join(@data_dir, "terrain.pmtiles") refute_path_exists File.join(@data_dir, "terrain.pmtiles")
@@ -195,12 +238,13 @@ class PmtilesBuilderTest < Minitest::Test
end end
def test_build_terrain_raises_when_one_is_already_running def test_build_terrain_raises_when_one_is_already_running
select_regions("ES")
ENV["PMTILES_BIN"] = @pmtiles_ok ENV["PMTILES_BIN"] = @pmtiles_ok
gate = Queue.new gate = Queue.new
thread = Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { gate.pop }) thread = Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { gate.pop })
assert_raises(Maptool::PmtilesBuilder::BuildError) do assert_raises(Maptool::PmtilesBuilder::BuildError) do
Maptool::PmtilesBuilder.build_terrain([0, 0, 1, 1], data_dir: @data_dir, fetch_manifest: -> { "unused" }) Maptool::PmtilesBuilder.build_terrain(data_dir: @data_dir, fetch_manifest: -> { "unused" })
end end
ensure ensure
gate.push(fake_manifest) gate.push(fake_manifest)