Add in-app regeneration of offline.pmtiles and terrain.pmtiles
CI / check (push) Successful in 20s
CI / deploy (push) Successful in 1m52s

A logged-in user can now trigger, from a new "Offline data" sidebar
panel, what was previously a fully manual go-pmtiles workflow
(docs/development.adoc): regenerate either file for the current map
viewport, no area-picker (same "visible area is the area" convention
the Overpass POI plugin already uses).

- lib/maptool/pmtiles_builder.rb: two independent background jobs
  (Thread + Mutex-guarded status Hash, same shape as app.rb's own GNSS
  poller), each writing to a .tmp path and atomically renaming into
  place so a build never serves a partially-written file mid-run.
  offline.pmtiles is a single `pmtiles extract` against Protomaps'
  public daily basemap build (byte-range extraction, not a full
  ~120GB download). terrain.pmtiles has no equivalent public
  Terrarium-encoded archive to extract from, so it fetches individual
  Terrarium tiles itself, packs them into an intermediate MBTiles
  (new sqlite3 gem), then the same `pmtiles convert`.
- 4 new app.rb routes (POST .../build, GET .../build-status) plus
  size/mtime added to the existing availability endpoints.
- public/js/offline_data.js: new sidebar panel, polling build-status
  the same self-rescheduling-setTimeout shape livetrack.js already
  uses for its own refresh loop.
- Both network-heavy pieces (the Protomaps build index, per-tile
  fetches) are injectable, so test/test_pmtiles_builder.rb exercises
  the actual build pipeline against a fake `pmtiles` binary without
  hitting a real network, matching this suite's existing
  keep-it-offline convention for external services.
This commit is contained in:
randy committed 2026-09-11 01:25:08 +02:00
1 parent fba8ca10a2
commit cd56ebe0a1
14 files changed
+740 -15

No files matched your search

+48 -2
View File
@@ -12,6 +12,7 @@ require_relative "lib/maptool/livetrack_store"
require_relative "lib/maptool/livetrack_client"
require_relative "lib/maptool/kiwix"
require_relative "lib/maptool/plugins"
require_relative "lib/maptool/pmtiles_builder"
module Maptool
class App < Sinatra::Base
@@ -125,6 +126,12 @@ module Maptool
halt 401, json(error: "login required", login_url: "/login") unless logged_in?
end
def pmtiles_file_status(path)
return { available: false } unless File.exist?(path)
{ available: true, size: File.size(path), mtime: File.mtime(path).utc.iso8601 }
end
def callback_url
"#{request.base_url}/auth/callback"
end
@@ -204,7 +211,7 @@ module Maptool
end
get "/api/offline-tiles" do
json available: File.exist?(OFFLINE_PMTILES_PATH)
json pmtiles_file_status(OFFLINE_PMTILES_PATH)
end
# Served as a plain range-request-capable file, not decoded server-side:
@@ -216,11 +223,33 @@ module Maptool
send_file OFFLINE_PMTILES_PATH, type: "application/octet-stream"
end
# Regenerates offline.pmtiles for the given bbox (the current map
# viewport, same "visible area is the area" convention as the Overpass
# POI plugin) by extracting it from Protomaps' own daily full-planet
# basemap build — see lib/maptool/pmtiles_builder.rb. Runs in the
# background; poll build-status below for progress.
post "/api/offline-tiles/build" do
require_login!
bbox = %w[south west north east].map { |key| params[key].to_s }
halt 400, json(error: "south, west, north, east are required") if bbox.any?(&:empty?)
begin
PmtilesBuilder.build_offline(bbox.map(&:to_f), data_dir: DATA_DIR)
rescue PmtilesBuilder::BuildError => e
halt 409, json(error: e.message)
end
json status: "started"
end
get "/api/offline-tiles/build-status" do
json PmtilesBuilder.offline_status
end
# Same optional-local-file story as offline.pmtiles above — see
# docs/development.adoc for how to generate one, until then hillshade
# just stays unavailable while the offline basemap is active.
get "/api/terrain-tiles" do
json available: File.exist?(TERRAIN_PMTILES_PATH)
json pmtiles_file_status(TERRAIN_PMTILES_PATH)
end
get "/tiles/terrain.pmtiles" do
@@ -229,6 +258,23 @@ module Maptool
send_file TERRAIN_PMTILES_PATH, type: "application/octet-stream"
end
post "/api/terrain-tiles/build" do
require_login!
bbox = %w[south west north east].map { |key| params[key].to_s }
halt 400, json(error: "south, west, north, east are required") if bbox.any?(&:empty?)
begin
PmtilesBuilder.build_terrain(bbox.map(&:to_f), data_dir: DATA_DIR)
rescue PmtilesBuilder::BuildError => e
halt 409, json(error: e.message)
end
json status: "started"
end
get "/api/terrain-tiles/build-status" do
json PmtilesBuilder.terrain_status
end
# Pinged once per page load (public/js/map.js) — url is a same-origin
# path (not kiwix-serve's own address, which the browser can't reach
# directly behind the reverse proxy in front of this app), proxied by