Replace offline basemap pipeline with Planetiler, unify online/offline styling

Offline tiles were built from Protomaps' own basemap schema, needing a
completely separate hand-tuned theme (protomaps-themes-base + OFFLINE_THEME)
from the online Liberty style's ONLINE_STYLE_PATCH — the two could never be
made to match exactly no matter how much either side got tuned (today's
session found and fixed several mismatches: building opacity, park
boundaries, road casings, road widths).

Offline.pmtiles is now built with Planetiler's default OpenMapTiles profile
against Geofabrik per-country OSM extracts (osmium-merged when more than one
country is selected), the same vector schema Liberty already expects.
offlineStyle() now clones the same Liberty JSON (vendored as a local
snapshot for reliability with no internet) and applies the identical
ONLINE_STYLE_PATCH, just repointed at local PMTiles/glyphs/sprite — one
palette instead of two, correct on both sides by construction.

Also vendors Liberty's icon sprite for the first time (offline had none
before), adds Java/osmium-tool/planetiler.jar to the Docker image, and
removes the now-unused protomaps-themes-base dependency and the
theme-comparison debug page that found today's bugs.

lib/maptool/pmtiles_builder.rb

@@ -9,12 +9,10 @@ module Maptool
   # Regenerates data/offline.pmtiles and data/terrain.pmtiles for the
   # currently *selected* regions (data/offline_regions.json, an array of
   # Natural Earth iso_a2 country codes — see available_regions/
-  # selected_regions/set_selected_regions below) by shelling out to
-  # go-pmtiles — the same tool docs/development.adoc already documents
-  # running by hand. No job queue: each build runs on its own background
-  # Thread and reports into a Mutex-guarded status Hash a route can poll,
-  # the same shape app.rb's own GNSS poller (GNSS_LATEST/GNSS_LATEST_MUTEX)
-  # already uses.
+  # selected_regions/set_selected_regions below). No job queue: each build
+  # runs on its own background Thread and reports into a Mutex-guarded
+  # status Hash a route can poll, the same shape app.rb's own GNSS poller
+  # (GNSS_LATEST/GNSS_LATEST_MUTEX) already uses.
   module PmtilesBuilder
     class BuildError < StandardError; end
 
@@ -22,24 +20,71 @@ module Maptool
     # countries, filtered to Europe and trimmed to name/iso_a2/iso_a3 —
     # see the plan this was built from for the exact filter. 110m is
     # deliberately coarse (a km or so of border precision, not
-    # survey-grade) — fine for "which tiles to grab," a same-schema swap
+    # survey-grade) — fine for "which country to grab," a same-schema swap
     # to Natural Earth's 50m/10m cut if that ever needs to be tighter.
     REGIONS_FILE = File.join(__dir__, "data", "europe_countries.geojson")
 
-    # Protomaps' own daily full-planet basemap build index — extract_offline
-    # below only pulls the HTTP byte ranges covering the requested region
-    # (go-pmtiles's `extract` supports a remote source over range requests),
-    # never the whole ~120GB file.
-    PROTOMAPS_BUILDS_INDEX = "https://build-metadata.protomaps.dev/builds.json"
-    PROTOMAPS_BUILD_BASE = "https://build.protomaps.com"
+    # Every iso_a2 code in REGIONS_FILE mapped to its Geofabrik download
+    # slug (https://download.geofabrik.de/<slug>-latest.osm.pbf) — Geofabrik
+    # names regions, not ISO codes, and doesn't always split the way a
+    # country border does, so this table exists rather than deriving the
+    # slug from the country name. Two known quirks worth knowing about,
+    # not worth working around:
+    #  - RU has no "europe/" prefix (Russia is its own top-level index,
+    #    and unlike every other entry here is a single ~4GB file covering
+    #    the whole country rather than a European-only slice).
+    #  - GB's "great-britain" slug is England/Scotland/Wales only —
+    #    Northern Ireland lives in "ireland-and-northern-ireland" instead
+    #    (bundled with IE, not GB). Selecting GB alone won't include NI.
+    GEOFABRIK_SLUGS = {
+      "AL" => "europe/albania",
+      "AT" => "europe/austria",
+      "BY" => "europe/belarus",
+      "BE" => "europe/belgium",
+      "BA" => "europe/bosnia-herzegovina",
+      "BG" => "europe/bulgaria",
+      "HR" => "europe/croatia",
+      "CZ" => "europe/czech-republic",
+      "DK" => "europe/denmark",
+      "EE" => "europe/estonia",
+      "FI" => "europe/finland",
+      "FR" => "europe/france",
+      "DE" => "europe/germany",
+      "GR" => "europe/greece",
+      "HU" => "europe/hungary",
+      "IS" => "europe/iceland",
+      "IE" => "europe/ireland-and-northern-ireland",
+      "IT" => "europe/italy",
+      "XK" => "europe/kosovo",
+      "LV" => "europe/latvia",
+      "LT" => "europe/lithuania",
+      "LU" => "europe/luxembourg",
+      "MD" => "europe/moldova",
+      "ME" => "europe/montenegro",
+      "NL" => "europe/netherlands",
+      "MK" => "europe/macedonia",
+      "NO" => "europe/norway",
+      "PL" => "europe/poland",
+      "PT" => "europe/portugal",
+      "RO" => "europe/romania",
+      "RU" => "russia",
+      "RS" => "europe/serbia",
+      "SK" => "europe/slovakia",
+      "SI" => "europe/slovenia",
+      "ES" => "europe/spain",
+      "SE" => "europe/sweden",
+      "CH" => "europe/switzerland",
+      "UA" => "europe/ukraine",
+      "GB" => "europe/great-britain",
+    }.freeze
 
     # Mapterhorn (a Protomaps-affiliated project) publishes Terrarium-
     # encoded terrain as pre-built PMTiles archives: one global
     # planet.pmtiles (z0-12) plus ~450 regional chunks (z13 up to a
-    # per-region max, 13-18) — same "extract only the region's byte ranges
-    # from a remote archive" story as offline.pmtiles above, so no more
-    # fetching individual tiles or building an MBTiles file ourselves the
-    # way an earlier version of this file did.
+    # per-region max, 13-18) — extract_offline below no longer touches
+    # Protomaps at all (see GEOFABRIK_SLUGS above), but terrain stays on
+    # this same "extract only the region's byte ranges from a remote
+    # archive" story via go-pmtiles, unrelated to the basemap swap.
     MAPTERHORN_MANIFEST_URL = "https://download.mapterhorn.com/download_urls.json"
     MAPTERHORN_PLANET_NAME = "planet.pmtiles"
     # Matches the online hillshade layer's own cap (public/js/overlays.js's
@@ -102,28 +147,22 @@ module Maptool
     end
     private_class_method :region_features
 
-    # Writes the FeatureCollection of just the selected countries into
-    # `dir` — this *is* the file go-pmtiles' `--region=` reads. No
-    # union/dissolve needed: go-pmtiles treats a FeatureCollection's
-    # features as one combined MultiPolygon on its own (confirmed against
-    # its own source, pmtiles/region.go's UnmarshalRegion). Also returns a
-    # plain aggregate [south,west,north,east] (min/max over every ring
-    # coordinate, no geometry library) used only for the Mapterhorn
-    # chunk-intersection check below — approximate is fine there, the
-    # `--region=` file is what does the real precise selection.
-    def self.region_geojson_and_bbox(codes, dir)
+    # Still used by terrain (below) to compute the aggregate bbox Mapterhorn
+    # chunk intersection needs — offline.pmtiles no longer needs this at
+    # all, since each Geofabrik country file is already clipped to that
+    # country's real border (more accurate than this 110m polygon, not
+    # less), so there's no separate "--region=" precision step for it
+    # anymore.
+    def self.region_bbox(codes)
       selected = region_features.select { |f| codes.include?(f["properties"]["iso_a2"]) }
       raise BuildError, "no matching regions for #{codes.join(", ")}" if selected.empty?
 
-      path = File.join(dir, "region.geojson")
-      File.write(path, JSON.generate(type: "FeatureCollection", features: selected))
-
       coords = selected.flat_map { |f| ring_coordinates(f["geometry"]) }
       lons = coords.map { |lng, _lat| lng }
       lats = coords.map { |_lng, lat| lat }
-      [path, [lats.min, lons.min, lats.max, lons.max]]
+      [lats.min, lons.min, lats.max, lons.max]
     end
-    private_class_method :region_geojson_and_bbox
+    private_class_method :region_bbox
 
     def self.ring_coordinates(geometry)
       case geometry["type"]
@@ -140,21 +179,21 @@ module Maptool
     # finishes — raises BuildError (the caller turns this into an error
     # response) if nothing's selected yet, or one's already running.
     #
-    # fetch_build_url exists as an injectable seam so tests can exercise the
-    # rest of this (temp path, go-pmtiles invocation, atomic rename) without
-    # a real network call — same reasoning test_helper.rb already gives for
-    # keeping /api/plugins route tests off the real Overpass API.
-    def self.build_offline(data_dir:, fetch_build_url: method(:latest_protomaps_build_url))
+    # download_extract exists as an injectable seam so tests can exercise
+    # the rest of this (scratch dir, osmium/planetiler invocation, atomic
+    # rename) without a real network call — same reasoning test_helper.rb
+    # already gives for keeping /api/plugins route tests off the real
+    # Overpass API.
+    def self.build_offline(data_dir:, download_extract: method(:download_extract))
       codes = selected_regions(data_dir)
       raise BuildError, "select at least one region first" if codes.empty?
 
       begin_run!(OFFLINE_MUTEX, OFFLINE_STATUS)
-      Thread.new { run_job(OFFLINE_MUTEX, OFFLINE_STATUS) { extract_offline(codes, data_dir, fetch_build_url) } }
+      Thread.new { run_job(OFFLINE_MUTEX, OFFLINE_STATUS) { extract_offline(codes, data_dir, download_extract) } }
     end
 
-    # fetch_manifest: same injectable-seam reasoning as fetch_build_url
-    # above, for the one Mapterhorn manifest lookup this needs instead of
-    # the Protomaps build index.
+    # fetch_manifest: same injectable-seam reasoning as download_extract
+    # above, for the one Mapterhorn manifest lookup this needs.
     def self.build_terrain(data_dir:, fetch_manifest: method(:fetch_mapterhorn_manifest))
       codes = selected_regions(data_dir)
       raise BuildError, "select at least one region first" if codes.empty?
@@ -181,56 +220,103 @@ module Maptool
 
     # --- offline.pmtiles ---
 
-    def self.extract_offline(codes, data_dir, fetch_build_url)
-      source = fetch_build_url.call
+    # One Geofabrik .osm.pbf per selected country (already clipped to that
+    # country's real border), osmium-merged into one file if more than one
+    # was selected (skipped entirely for a single country — nothing to
+    # merge), then tiled by Planetiler using its default OpenMapTiles
+    # profile straight to PMTiles — the same vector schema
+    # tiles.openfreemap.org's Liberty style already expects, so
+    # public/js/map.js's offlineStyle() can reuse ONLINE_STYLE_PATCH
+    # unchanged instead of a separate protomaps-themes-base palette.
+    def self.extract_offline(codes, data_dir, download_extract)
       scratch_dir = Dir.mktmpdir("maptool-offline")
-      region_path, = region_geojson_and_bbox(codes, scratch_dir)
-      tmp_path = File.join(data_dir, "offline.pmtiles.tmp")
+      sources = codes.map do |code|
+        slug = GEOFABRIK_SLUGS.fetch(code) { raise BuildError, "no Geofabrik source known for region #{code}" }
+        dest = File.join(scratch_dir, "#{code}.osm.pbf")
+        last_modified = download_extract.call(geofabrik_url(slug), dest)
+        { "code" => code, "slug" => slug, "last_modified" => last_modified, "path" => dest }
+      end
 
-      run_pmtiles!("extract", source, tmp_path, "--region=#{region_path}", "--maxzoom=15")
+      osm_path =
+        if sources.size == 1
+          sources.first["path"]
+        else
+          merged_path = File.join(scratch_dir, "merged.osm.pbf")
+          run_osmium!("merge", *sources.map { |s| s["path"] }, "-o", merged_path)
+          merged_path
+        end
+
+      tmp_path = File.join(data_dir, "offline.pmtiles.tmp")
+      run_planetiler!("--osm-path=#{osm_path}", "--output=#{tmp_path}")
       File.rename(tmp_path, File.join(data_dir, "offline.pmtiles"))
-      File.write(offline_meta_path(data_dir), JSON.generate(source: File.basename(URI(source).path)))
+      File.write(offline_meta_path(data_dir), JSON.generate(sources: sources.map { |s| s.slice("code", "slug", "last_modified") }))
     ensure
       FileUtils.remove_entry(scratch_dir) if scratch_dir && Dir.exist?(scratch_dir)
       File.delete(tmp_path) if tmp_path && File.exist?(tmp_path)
     end
     private_class_method :extract_offline
 
-    def self.latest_protomaps_build_url
-      response = Net::HTTP.get_response(URI(PROTOMAPS_BUILDS_INDEX))
-      raise BuildError, "could not reach #{PROTOMAPS_BUILDS_INDEX}" unless response.is_a?(Net::HTTPSuccess)
-
-      builds = JSON.parse(response.body)
-      raise BuildError, "protomaps build index was empty" if builds.empty?
-
-      "#{PROTOMAPS_BUILD_BASE}/#{builds.last.fetch("key")}"
-    rescue JSON::ParserError
-      raise BuildError, "protomaps build index was not valid JSON"
+    def self.geofabrik_url(slug)
+      "https://download.geofabrik.de/#{slug}-latest.osm.pbf"
     end
-    private_class_method :latest_protomaps_build_url
+    private_class_method :geofabrik_url
 
-    # Sidecar recording which Protomaps build (a date-stamped filename, e.g.
-    # "20260910.pmtiles") the on-disk offline.pmtiles was actually extracted
-    # from — the file itself carries no such record. nil if never built (or
-    # built before this existed).
+    # Real network implementation of the download_extract seam above —
+    # streams the extract to dest and returns its Last-Modified header
+    # (free off the same GET, no separate HEAD needed) for the sidecar
+    # meta / update-check below.
+    def self.download_extract(url, dest)
+      uri = URI(url)
+      response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https") { |http| http.get(uri) }
+      raise BuildError, "could not download #{url} (HTTP #{response.code})" unless response.is_a?(Net::HTTPSuccess)
+
+      File.binwrite(dest, response.body)
+      response["last-modified"]
+    end
+
+    # Sidecar recording which Geofabrik sources (country code, slug, and
+    # that request's Last-Modified) the on-disk offline.pmtiles was
+    # actually built from — the file itself carries no such record. nil if
+    # never built (or built before this existed).
     def self.offline_meta(data_dir)
       read_json(offline_meta_path(data_dir))
     end
 
-    # Compares the persisted build against the current latest without
-    # rebuilding anything — nil (not true/false) if there's nothing to
-    # compare yet (no local build) or the upstream index couldn't be
-    # reached, so the caller can tell "don't know" apart from "no".
-    def self.offline_update_available?(data_dir, fetch_build_url: method(:latest_protomaps_build_url))
+    # Compares each currently-selected country's live Geofabrik
+    # Last-Modified against what's recorded, without downloading anything
+    # — nil (not true/false) if there's nothing to compare yet (no local
+    # build, nothing selected) or a check couldn't be completed, so the
+    # caller can tell "don't know" apart from "no". Also naturally reports
+    # "yes" if the selection changed since the last build (a country now
+    # relevant that wasn't recorded, or vice versa) — same reasoning
+    # terrain_update_available? below already uses.
+    def self.offline_update_available?(data_dir, fetch_last_modified: method(:fetch_geofabrik_last_modified))
       meta = offline_meta(data_dir)
       return nil unless meta
 
-      latest = File.basename(URI(fetch_build_url.call).path)
-      latest != meta["source"]
+      codes = selected_regions(data_dir)
+      return nil if codes.empty?
+
+      current = codes.map do |code|
+        slug = GEOFABRIK_SLUGS[code]
+        return nil unless slug
+
+        { "code" => code, "last_modified" => fetch_last_modified.call(geofabrik_url(slug)) }
+      end.sort_by { |s| s["code"] }
+      recorded = (meta["sources"] || []).map { |s| s.slice("code", "last_modified") }.sort_by { |s| s["code"] }
+      current != recorded
     rescue BuildError
       nil
     end
 
+    def self.fetch_geofabrik_last_modified(url)
+      uri = URI(url)
+      response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https") { |http| http.head(uri) }
+      raise BuildError, "could not reach #{url}" unless response.is_a?(Net::HTTPSuccess)
+
+      response["last-modified"]
+    end
+
     def self.offline_meta_path(data_dir)
       File.join(data_dir, "offline.pmtiles.meta.json")
     end
@@ -258,8 +344,10 @@ module Maptool
       raise BuildError, "mapterhorn manifest is missing #{MAPTERHORN_PLANET_NAME}" unless planet
 
       scratch_dir = Dir.mktmpdir("maptool-terrain")
-      region_path, bbox = region_geojson_and_bbox(codes, scratch_dir)
-      south, west, north, east = bbox
+      south, west, north, east = region_bbox(codes)
+      region_path = File.join(scratch_dir, "region.geojson")
+      selected = region_features.select { |f| codes.include?(f["properties"]["iso_a2"]) }
+      File.write(region_path, JSON.generate(type: "FeatureCollection", features: selected))
 
       chunks = items.select { |item| item["name"] != MAPTERHORN_PLANET_NAME && bbox_intersects?(item, south, west, north, east) }
 
@@ -322,13 +410,10 @@ module Maptool
       planet = items.find { |item| item["name"] == MAPTERHORN_PLANET_NAME }
       return nil unless planet
 
-      Dir.mktmpdir do |dir|
-        _region_path, bbox = region_geojson_and_bbox(codes, dir)
-        south, west, north, east = bbox
-        chunks = items.select { |item| item["name"] != MAPTERHORN_PLANET_NAME && bbox_intersects?(item, south, west, north, east) }
-        current = [planet, *chunks].map { |i| { "name" => i["name"], "md5sum" => i["md5sum"] } }.sort_by { |s| s["name"] }
-        current != (meta["sources"] || []).sort_by { |s| s["name"] }
-      end
+      south, west, north, east = region_bbox(codes)
+      chunks = items.select { |item| item["name"] != MAPTERHORN_PLANET_NAME && bbox_intersects?(item, south, west, north, east) }
+      current = [planet, *chunks].map { |i| { "name" => i["name"], "md5sum" => i["md5sum"] } }.sort_by { |s| s["name"] }
+      current != (meta["sources"] || []).sort_by { |s| s["name"] }
     rescue BuildError
       nil
     end
@@ -347,12 +432,24 @@ module Maptool
     end
     private_class_method :read_json
 
-    # ENV read lazily (not a load-time constant) so tests can point this at
-    # a fake script without a require-order dance.
+    # ENV read lazily (not load-time constants) so tests can point these at
+    # fake binaries without a require-order dance.
     def self.pmtiles_bin
       ENV.fetch("PMTILES_BIN", "pmtiles")
     end
 
+    def self.osmium_bin
+      ENV.fetch("OSMIUM_BIN", "osmium")
+    end
+
+    def self.java_bin
+      ENV.fetch("JAVA_BIN", "java")
+    end
+
+    def self.planetiler_jar
+      ENV.fetch("PLANETILER_JAR", "/usr/local/lib/planetiler.jar")
+    end
+
     def self.run_pmtiles!(*args)
       _stdout, stderr, status = Open3.capture3(pmtiles_bin, *args)
       raise BuildError, "pmtiles exited with status #{status.exitstatus}: #{stderr.strip}" unless status.success?
@@ -360,5 +457,21 @@ module Maptool
       raise BuildError, "could not run '#{pmtiles_bin}' (#{e.message}) — is go-pmtiles installed and on PATH? See docs/development.adoc."
     end
     private_class_method :run_pmtiles!
+
+    def self.run_osmium!(*args)
+      _stdout, stderr, status = Open3.capture3(osmium_bin, *args)
+      raise BuildError, "osmium exited with status #{status.exitstatus}: #{stderr.strip}" unless status.success?
+    rescue Errno::ENOENT, Errno::EACCES => e
+      raise BuildError, "could not run '#{osmium_bin}' (#{e.message}) — is osmium-tool installed and on PATH? See docs/development.adoc."
+    end
+    private_class_method :run_osmium!
+
+    def self.run_planetiler!(*args)
+      _stdout, stderr, status = Open3.capture3(java_bin, "-jar", planetiler_jar, *args)
+      raise BuildError, "planetiler exited with status #{status.exitstatus}: #{stderr.strip}" unless status.success?
+    rescue Errno::ENOENT, Errno::EACCES => e
+      raise BuildError, "could not run '#{java_bin} -jar #{planetiler_jar}' (#{e.message}) — is Java and planetiler.jar installed? See docs/development.adoc."
+    end
+    private_class_method :run_planetiler!
   end
 end