diff --git a/TODO.adoc b/TODO.adoc index 6c467af..8393d04 100644 --- a/TODO.adoc +++ b/TODO.adoc @@ -13,8 +13,7 @@ independence from connectivity. ** [ ] Terrain slope readout for parking (numeric, not just visual shading) * [ ] Contour lines from OSM for hikes and mountain passes * [x] Hiking trail overlay — Waymarked Trails raster tiles online (`public/js/hiking_trails.js`), colored per-route by the OSM `colour` tag they already render (the KST/KČT paint-blaze colors in Slovakia/Czechia, plain red elsewhere). Click queries a one-off Overpass `around` search (racing 3 public mirrors) for the route's name/colour/network/operator/distance -** [x] Offline copy wired in for real, no custom Planetiler profile needed — `Maptool::HikingTrails` (`lib/maptool/hiking_trails.rb`) runs `osmium tags-filter r/route=hiking` + `add-locations-to-ways` + `cat -f opl` against the *same* per-country Geofabrik `.osm.pbf` already downloaded for `offline.pmtiles`, no new download. Generated automatically as the last step of every offline build (`PmtilesBuilder.extract_hiking_trails`, best-effort — a failure here doesn't fail the whole build) into `data/hiking_trails.geojson`, served at `GET /tiles/hiking_trails.geojson` only once the overlay's actually toggled on. Offline click uses a local `queryRenderedFeatures` hit-test instead of Overpass — instant, no network. See docs/development.adoc -*** [ ] File size for a many-country selection could get large (Slovakia alone: ~3000 routes) — coordinates are rounded to 5dp but there's no real tiling/simplification yet; revisit if a big selection makes the browser fetch noticeably slow +** [x] Offline copy wired in for real, no custom Planetiler profile needed — `Maptool::HikingTrails` (`lib/maptool/hiking_trails.rb`) runs `osmium tags-filter r/route=hiking` + `add-locations-to-ways` + `cat -f opl` against the *same* per-country Geofabrik `.osm.pbf` already downloaded for `offline.pmtiles`, no new download. One `data/hiking_trails/.geojson` per country (not one combined file), generated as the last step of every offline build and cached like `geofabrik_cache` (skipped once already newer than its source PBF) — a many-country selection no longer means one giant file, since `GET /api/hiking-trails` lists available countries and the Overlays panel's own checklist only fetches `GET /tiles/hiking_trails/.geojson` for whichever ones get ticked. Offline click uses a local `queryRenderedFeatures` hit-test instead of Overpass — instant, no network. See docs/development.adoc * [x] Overpass/OSM scripts to extract POIs (drinking water, dump stations, LPG, public toilets, laundromats, shops, springs) — `rake overpass:pois`, see docs/development.adoc; writes a `.geojson` to drop in `data/gpx/` or upload. Also runnable from inside the app itself now, see the import plugins entry below (`plugins/overpass-pois` wraps this same query). Each category now carries a real `POI_ICONS` glyph and color (not p4n2gpx's plain red marker) plus a popup description built from opening_hours/operator/website tags * [x] Import plugins: a pluggable way to add more data sources (Park4Night, iOverlander, Freecampsites, ...) without their scrapers necessarily living in this (public) repo — sidebar icon (logged in only) lists whatever's under `plugins/`, "Run for this area" uses the current map viewport as the bbox, no separate area-picker needed. Contract: a `plugin.json` manifest + a script that reads `--south`/`--west`/`--north`/`--east` and prints a GeoJSON `FeatureCollection` to stdout (`lib/maptool/plugins.rb`, `GET /api/plugins`, `POST /api/plugins/:name/run`) — output saved the same way an upload would be. `plugins/overpass-pois` ships as the reference example; a private/ToS-sensitive scraper just drops its own directory in locally and is never `git add`ed ** [x] Park4Night — private plugin (`data/plugins/p4n2gpx/`, outside this repo per the "ToS-sensitive scraper" note above); iOverlander/Freecampsites still open, same story diff --git a/app.rb b/app.rb index 50f0423..f087b58 100644 --- a/app.rb +++ b/app.rb @@ -62,7 +62,6 @@ module Maptool OFFLINE_PMTILES_PATH = File.join(DATA_DIR, "offline.pmtiles") TERRAIN_PMTILES_PATH = File.join(DATA_DIR, "terrain.pmtiles") - HIKING_TRAILS_PATH = PmtilesBuilder.hiking_trails_path(DATA_DIR) GPX_STORE = GpxStore.new(File.join(DATA_DIR, "gpx")) LIVETRACK_STORE = LiveTrackStore.new(File.join(DATA_DIR, "livetrack.json")) @@ -361,24 +360,28 @@ module Maptool # PmtilesBuilder.extract_hiking_trails) rather than its own separate # build — it's derived from the exact same per-country .osm.pbf cache # offline.pmtiles itself already downloads, so there's nothing of its - # own to trigger or delete; DELETE /api/offline-tiles removes this - # file too. No `meta` argument here (unlike offline/terrain above) — - # parsing the whole GeoJSON just to report a feature count would cost - # real time on every page load's status check for what's a multi-MB - # file across several countries; size/mtime alone is enough to know - # whether it's there. + # own to trigger; DELETE /api/offline-tiles removes it too. One file + # per country (not one combined file) so a viewer only ever fetches + # the country they actually pick in the Hiking trails overlay's own + # checklist (public/js/hiking_trails.js), not every selected country's + # trails at once — `countries` lists which ones exist to build that + # checklist from. get "/api/hiking-trails" do - json pmtiles_file_status(HIKING_TRAILS_PATH, nil) + json countries: PmtilesBuilder.hiking_trails_countries(DATA_DIR) end - # Fetched client-side only when the Hiking trails overlay is actually - # toggled on (public/js/hiking_trails.js), not on page load — same - # "on demand" story as the online raster tiles it replaces once this - # is available. - get "/tiles/hiking_trails.geojson" do - halt 404 unless File.exist?(HIKING_TRAILS_PATH) + # Fetched client-side only for a country the viewer actually ticks in + # the Hiking trails checklist, not on page load. :code is constrained + # to exactly what hiking_trails_countries can ever return (two + # uppercase letters) — otherwise this would accept an arbitrary + # ".."-laden path segment. + get "/tiles/hiking_trails/:code.geojson" do + halt 404 unless params[:code].match?(/\A[A-Z]{2}\z/) - send_file HIKING_TRAILS_PATH, type: "application/geo+json" + path = PmtilesBuilder.hiking_trails_country_path(DATA_DIR, params[:code]) + halt 404 unless File.exist?(path) + + send_file path, type: "application/geo+json" end # Pinged once per page load (public/js/map.js) — url is a same-origin diff --git a/docs/development.adoc b/docs/development.adoc index 46f952e..13b31a7 100644 --- a/docs/development.adoc +++ b/docs/development.adoc @@ -106,11 +106,12 @@ size rather than assuming one, so this isn't a problem either way. The sidebar's "Hiking trails" overlay works online out of the box (waymarkedtrails.org raster tiles, `public/js/hiking_trails.js`) — clicking a trail fires a one-off Overpass query for its name/colour/ -network/distance. For it to work offline too, `data/hiking_trails.geojson` -is generated automatically as the last step of every `offline.pmtiles` -build (`Maptool::HikingTrails`, `lib/maptool/hiking_trails.rb`), straight -from the *same* per-country `.osm.pbf` files already downloaded for -`offline.pmtiles` itself — no separate download, no third-party service: +network/distance. For it to work offline too, one `data/hiking_trails/ +.geojson` per country is generated automatically as the last step +of every `offline.pmtiles` build (`Maptool::HikingTrails`, +`lib/maptool/hiking_trails.rb`), straight from the *same* per-country +`.osm.pbf` files already downloaded for `offline.pmtiles` itself — no +separate download, no third-party service: [source,shell] ---- @@ -130,20 +131,30 @@ tool stitches a route relation's member ways into one line for you waymarkedtrails.org and this app's own online Overpass click handler already do. -Best-effort: a hiking-trails extraction problem on one country's PBF -logs a warning and is skipped rather than failing the whole -`offline.pmtiles` build over what's a bonus overlay, not the core map -(`PmtilesBuilder.extract_hiking_trails`). Coordinates are rounded to 5 -decimal places (~1m, plenty for a rendered line) to keep the combined -file smaller — for a many-country selection it can still be a sizeable -GeoJSON (Slovakia alone: ~3000 routes, ~30MB unrounded), fetched -client-side only once the overlay is actually toggled on, not on page -load. `rake hiking:trails[pbf,out]` runs the extraction by hand against -one `.osm.pbf` for testing. +Best-effort per country: an extraction problem on one country's PBF logs +a warning and is skipped rather than failing the whole `offline.pmtiles` +build, or even the other countries in the same selection, over what's a +bonus overlay, not the core map (`PmtilesBuilder.extract_hiking_trails`). +Cached the same way `geofabrik_cache` is (skipped once a country's own +extract is already newer than its source `.osm.pbf`), so growing the +region selection by one country only extracts that one. Coordinates are +rounded to 5 decimal places (~1m, plenty for a rendered line) — Slovakia +alone is ~3000 routes, ~30MB unrounded. -`DELETE /api/offline-tiles` removes this file along with `offline.pmtiles` -and the Geofabrik cache — there's no separate delete/rebuild route for it, -since it's not an independent build. +One file per country rather than one combined file: `GET /api/hiking-trails` +lists which country codes have an extract (`countries: [...]`), and the +Overlays panel's "Hiking trails" item turns that into a checklist — +each ticked country fetches its own `GET /tiles/hiking_trails/.geojson` +only then, not on page load and not for countries never ticked. Offline +click hit-tests the loaded vector layer(s) directly +(`queryRenderedFeatures`) instead of querying Overpass, so it resolves +instantly with no network at all. `rake hiking:trails[pbf,out]` runs the +extraction by hand against one `.osm.pbf` for testing. + +`DELETE /api/offline-tiles` removes the whole `data/hiking_trails/` +directory along with `offline.pmtiles` and the Geofabrik cache — there's +no separate delete/rebuild route for it, since it's not an independent +build. == In-app regeneration diff --git a/lib/maptool/hiking_trails.rb b/lib/maptool/hiking_trails.rb index 124040b..c891fcf 100644 --- a/lib/maptool/hiking_trails.rb +++ b/lib/maptool/hiking_trails.rb @@ -63,22 +63,6 @@ module Maptool end end - # Runs extract() over one .osm.pbf per selected country and combines the - # results into a single FeatureCollection — a route relation crossing a - # border (an E-path, say) shows up once per country whose extract it - # was pulled from, deduplicated here by OSM relation id (properties.id) - # rather than left to render as overlapping duplicate lines. - # extract: same injectable-seam reasoning as PmtilesBuilder's own - # download_extract:/fetch_manifest: — lets a test swap out the real - # (osmium-shelling-out) implementation for a canned one. - def self.extract_merged(pbf_paths, extract: method(:extract)) - seen = {} - pbf_paths.each do |pbf_path| - extract.call(pbf_path)["features"].each { |feature| seen[feature["properties"]["id"]] ||= feature } - end - { "type" => "FeatureCollection", "features" => seen.values } - end - # Pure — no subprocess — so it's cheaply testable against a canned OPL # fixture, same "keep the real logic testable without the real tool # installed" reasoning OverpassPois.to_geojson already gives for staying diff --git a/lib/maptool/pmtiles_builder.rb b/lib/maptool/pmtiles_builder.rb index 264cd0d..47bcb58 100644 --- a/lib/maptool/pmtiles_builder.rb +++ b/lib/maptool/pmtiles_builder.rb @@ -299,26 +299,55 @@ module Maptool end private_class_method :extract_offline - # Runs over the same per-country .osm.pbf files sources above already + # One GeoJSON per country (data/hiking_trails/.geojson), not one + # merged file — the frontend only ever needs to show a country at a + # time the viewer actually picks (see HikingTrailsControl's country + # checklist), and a combined multi-country file was needlessly making + # every viewer fetch every selected country's trails regardless. + # + # Runs over the same per-country .osm.pbf files `sources` above already # downloaded/cached for offline.pmtiles itself — no separate download. - # Best-effort: a hiking-trails extraction problem (a malformed PBF - # region, osmium hiccup on one particular file) shouldn't fail the - # whole offline.pmtiles build over what's a bonus overlay, not the - # core map — logged into the sidecar meta's own "error" field instead, - # so it's visible without taking build_offline itself down. + # Cached the same way geofabrik_cache is: skipped for a country whose + # extract is already newer than its source .osm.pbf, so growing the + # region selection by one country only extracts that one, not every + # already-covered country all over again. + # + # Best-effort per country: an extraction problem (a malformed PBF + # region, an osmium hiccup) shouldn't fail the whole offline.pmtiles + # build, or even the other countries in this same selection, over + # what's a bonus overlay, not the core map. def self.extract_hiking_trails(sources, data_dir) - geojson = HikingTrails.extract_merged(sources.map { |s| s["path"] }) - tmp_path = "#{hiking_trails_path(data_dir)}.tmp" - File.write(tmp_path, JSON.generate(geojson)) - File.rename(tmp_path, hiking_trails_path(data_dir)) - rescue HikingTrails::ExtractError => e - File.delete(tmp_path) if tmp_path && File.exist?(tmp_path) - warn "hiking trails extraction failed, skipping: #{e.message}" + FileUtils.mkdir_p(hiking_trails_dir(data_dir)) + sources.each do |source| + out_path = hiking_trails_country_path(data_dir, source["code"]) + next if File.exist?(out_path) && File.mtime(out_path) >= File.mtime(source["path"]) + + tmp_path = "#{out_path}.tmp" + begin + File.write(tmp_path, JSON.generate(HikingTrails.extract(source["path"]))) + File.rename(tmp_path, out_path) + rescue HikingTrails::ExtractError => e + File.delete(tmp_path) if File.exist?(tmp_path) + warn "hiking trails extraction failed for #{source["code"]}, skipping: #{e.message}" + end + end end private_class_method :extract_hiking_trails - def self.hiking_trails_path(data_dir) - File.join(data_dir, "hiking_trails.geojson") + def self.hiking_trails_dir(data_dir) + File.join(data_dir, "hiking_trails") + end + + def self.hiking_trails_country_path(data_dir, code) + File.join(hiking_trails_dir(data_dir), "#{code}.geojson") + end + + # Country codes with an already-extracted offline hiking-trails file, + # sorted — not necessarily the same as the *current* region selection + # (accumulates like geofabrik_cache does; a country deselected after + # being built once is left alone rather than swept up here). + def self.hiking_trails_countries(data_dir) + Dir.glob(File.join(hiking_trails_dir(data_dir), "*.geojson")).map { |path| File.basename(path, ".geojson") }.sort end # Renders a report_step string like "fetching PL (3/4) — done: SK, CZ — @@ -468,7 +497,7 @@ module Maptool File.delete(File.join(data_dir, "offline.pmtiles")) if File.exist?(File.join(data_dir, "offline.pmtiles")) File.delete(offline_meta_path(data_dir)) if File.exist?(offline_meta_path(data_dir)) - File.delete(hiking_trails_path(data_dir)) if File.exist?(hiking_trails_path(data_dir)) + FileUtils.remove_entry(hiking_trails_dir(data_dir)) if Dir.exist?(hiking_trails_dir(data_dir)) cache_dir = geofabrik_cache_dir(data_dir) FileUtils.remove_entry(cache_dir) if Dir.exist?(cache_dir) end diff --git a/public/css/style.css b/public/css/style.css index bcb0278..5d92969 100644 --- a/public/css/style.css +++ b/public/css/style.css @@ -1579,6 +1579,22 @@ html, body { font-size: 12px; } +.hiking-trail-countries { + display: flex; + flex-wrap: wrap; + gap: 4px 12px; + list-style: none; + margin: 0; + padding: 0; +} + +.hiking-trail-countries__item label { + display: flex; + align-items: center; + gap: 4px; + cursor: pointer; +} + .livetrack-panel__list { flex: 1; overflow-y: auto; diff --git a/public/js/hiking_trails.js b/public/js/hiking_trails.js index 77fa30e..9c4f745 100644 --- a/public/js/hiking_trails.js +++ b/public/js/hiking_trails.js @@ -4,9 +4,6 @@ import { firstSymbolLayerId } from "/js/radar.js"; import { escapeHtml } from "/js/gpx.js"; import { lineLength, formatDistance } from "/js/measure.js"; -const SOURCE_ID = "hiking-trails"; -const LAYER_ID = "hiking-trails-line"; - // Overpass has permissive CORS (verified: Access-Control-Allow-Origin: *), // so this queries it directly from the browser — no server-side proxy // needed, same "public API, no key" story as the Nominatim search already @@ -34,22 +31,36 @@ const SAFE_COLOR = /^(#[0-9a-fA-F]{3,8}|[a-zA-Z]+)$/; // Pre-rendered raster tiles, not vector — waymarkedtrails.org already // colors each route by its OSM `colour` tag (the KST/KČT-style paint-blaze // colors used in Slovakia/Czechia), falling back to plain red where a route -// has none, so there's no styling to do here ourselves. Used when there's -// no local offline extraction to draw from instead (see _addOfflineLayer -// below) — same "quietly renders nothing without a network" story as the -// Wikipedia/Radar/Contour lines overlays. +// has none, so there's no styling to do here ourselves. Used whenever no +// offline extract is available at all — same "quietly renders nothing +// without a network" story as the Wikipedia/Radar/Contour lines overlays. +const ONLINE_SOURCE_ID = "hiking-trails-online"; +const ONLINE_LAYER_ID = "hiking-trails-online-line"; const TILE_URL = "https://tile.waymarkedtrails.org/hiking/{z}/{x}/{y}.png"; const ATTRIBUTION = 'Trails © Waymarked Trails (CC-BY-SA), map data © OpenStreetMap contributors'; const OPACITY = 0.85; -// Offline: a real vector line layer, GeoJSON generated at build time by -// Maptool::HikingTrails straight from the same Geofabrik .osm.pbf -// offline.pmtiles itself is built from (see lib/maptool/hiking_trails.rb, -// PmtilesBuilder.extract_offline) — no network needed at all, and no -// waiting on Overpass for click details either (queried locally via -// queryRenderedFeatures instead, see _handleClick below). -const OFFLINE_GEOJSON_URL = "/tiles/hiking_trails.geojson"; +// Offline: one real vector line layer per country the viewer actually +// picks, GeoJSON generated at build time by Maptool::HikingTrails straight +// from the same Geofabrik .osm.pbf offline.pmtiles itself is built from +// (see lib/maptool/hiking_trails.rb, PmtilesBuilder.extract_hiking_trails) +// — no network needed, and no waiting on Overpass for click details either +// (queried locally via queryRenderedFeatures instead, see _handleClick +// below). One file per country rather than one combined file: a viewer +// covering a dozen countries' worth of offline map data rarely wants all +// of their hiking trails loaded at once, so this is opt-in per country +// instead. +function offlineSourceId(code) { + return `hiking-trails-offline-${code}`; +} +function offlineLayerId(code) { + return `hiking-trails-offline-${code}-line`; +} +function offlineUrl(code) { + return `/tiles/hiking_trails/${code}.geojson`; +} + // waymarkedtrails.org's own fallback for an untagged route. const DEFAULT_TRAIL_COLOR = "#c1272d"; // Real-world `colour` values seen in production data (see the 2026-09-13 @@ -66,68 +77,115 @@ function normalizedTrailColor(raw) { return /^#[0-9a-fA-F]{3,8}$/.test(trimmed) ? trimmed : DEFAULT_TRAIL_COLOR; } -// A plain on/off Overlays item, same shape as ContourControl — one static -// raster overlay, no scrub/forecast controls to expand into. +// An Overlays panel item — a plain on/off checkbox (raster overlay) until +// offline extracts exist for at least one country, at which point it +// becomes an expandable per-country checklist instead (same expand shape +// as RadarControl), since a viewer with a dozen countries' worth of +// offline map data rarely wants all of their hiking trails loaded, let +// alone parsed by the browser, at once. export class HikingTrailsControl { constructor(map) { this._map = map; - this._active = false; - this._offlineAvailable = false; - this._clickHandler = (event) => this._handleClick(event); + this._active = false; // the online-mode checkbox's own state + this._offlineCountries = []; // codes with an extract available on the server + this._selectedCountries = new Set(); // codes currently shown, offline mode + + map.on("click", (event) => this._handleClick(event)); // setStyle() (switching Online/Offline) tears down all custom sources/ - // layers — put this back if it was on, same as every other Overlays - // control. + // layers — put whatever was showing back, same as every other + // Overlays control. map.on("style.load", () => { - if (this._active) this._addLayer(); + if (this._active) this._addOnlineLayer(); + for (const code of this._selectedCountries) this._addOfflineLayer(code); }); } - // Called once map.js's GET /api/hiking-trails check resolves — same - // "prefer the local file whenever it exists" story as - // OverlaysControl#setHillshadeOfflineAvailable, independent of whether - // the Online or Offline *base map* happens to be selected right now. - setOfflineAvailable(available) { - this._offlineAvailable = available; - } - mount(li) { li.innerHTML = ` -