Compare commits

..
2 Commits
Author SHA1 Message Date
randy 079ac37787 Wire offline hiking trails into the real build, API, and renderer
CI / deploy (push) Successful in 12s
CI / check (push) Successful in 19s
data/hiking_trails.geojson now generates automatically as the last
step of every offline.pmtiles build (PmtilesBuilder.extract_hiking_trails,
best-effort — a failure here doesn't fail the whole build), merged
across all selected countries with cross-border dedup by OSM relation
id (HikingTrails.extract_merged). Served at GET /tiles/hiking_trails.geojson
and GET /api/hiking-trails, fetched client-side only once the overlay
is actually toggled on.

hiking_trails.js now prefers this local file whenever it exists,
regardless of which base map (online/offline) is active — same
"prefer local when available" story as HillshadeControl already uses
for terrain. Offline click hit-tests the real vector layer directly
(queryRenderedFeatures) instead of querying Overpass — instant, no
network. Coordinates are rounded to 5dp to keep the combined file
smaller; colour tag values are normalized against a known-safe set
before ever reaching a style paint property.

Verified extract_merged (new: combines multiple countries' extracts,
dedupes overlapping relations) against two real cached country PBFs
on brick directly, not just fixtures.
2026-09-13 10:31:59 +02:00
randy b35ace9734 Fix hiking trails extraction: add-locations-to-ways was missing
Verified live on brick against real cached extracts: a plain Geofabrik
PBF's ways only carry bare node references, not coordinates, so
locations_on_ways on the final `cat -f opl` had nothing to embed —
every way came out as e.g. n123xy, empty x/y, and every relation got
dropped for having no usable geometry (0 routes for Macedonia).

osmium add-locations-to-ways between the tags-filter and cat steps
resolves those references first. Re-verified after the fix: 45 routes
for Macedonia, 3009 for Slovakia, with real colour/ref/operator/
distance (326 red, 782 yellow, 510 blue, 902 green KST-style routes).
2026-09-13 09:26:22 +02:00
11 changed files with 387 additions and 31 deletions

No files matched your search

+3 -2
View File
@@ -12,8 +12,9 @@ independence from connectivity.
* [x] Shaded relief / DEM layer — hillshade toggle in the sidebar, sourced from public Terrarium tiles online; reads `data/terrain.pmtiles` for offline if one's been generated (see docs/development.adoc), otherwise the toggle is online-only for now * [x] Shaded relief / DEM layer — hillshade toggle in the sidebar, sourced from public Terrarium tiles online; reads `data/terrain.pmtiles` for offline if one's been generated (see docs/development.adoc), otherwise the toggle is online-only for now
** [ ] Terrain slope readout for parking (numeric, not just visual shading) ** [ ] Terrain slope readout for parking (numeric, not just visual shading)
* [ ] Contour lines from OSM for hikes and mountain passes * [ ] Contour lines from OSM for hikes and mountain passes
* [x] Hiking trail overlay — Waymarked Trails raster tiles (`public/js/hiking_trails.js`), colored per-route by the OSM `colour` tag they already render (the KST/KČT paint-blaze colors in Slovakia/Czechia, plain red elsewhere), so no styling logic needed here. Click queries a one-off Overpass `around` search (racing 3 public mirrors) for the route's name/colour/network/operator/distance, no server involved * [x] Hiking trail overlay — Waymarked Trails raster tiles online (`public/js/hiking_trails.js`), colored per-route by the OSM `colour` tag they already render (the KST/KČT paint-blaze colors in Slovakia/Czechia, plain red elsewhere). Click queries a one-off Overpass `around` search (racing 3 public mirrors) for the route's name/colour/network/operator/distance
** [x] Prototype: an offline extraction path exists after all, no custom Planetiler profile needed — `Maptool::HikingTrails.extract` (`lib/maptool/hiking_trails.rb`, `rake hiking:trails[pbf,out]`) runs `osmium tags-filter r/route=hiking` + `osmium cat -f opl` against the *same* Geofabrik `.osm.pbf` already downloaded for `offline.pmtiles` — no new download, no new dependency. Not wired into the actual offline build/rendering yet, still deferred until there's a route planner to pair it with (see the entry above) ** [x] Offline copy wired in for real, no custom Planetiler profile needed — `Maptool::HikingTrails` (`lib/maptool/hiking_trails.rb`) runs `osmium tags-filter r/route=hiking` + `add-locations-to-ways` + `cat -f opl` against the *same* per-country Geofabrik `.osm.pbf` already downloaded for `offline.pmtiles`, no new download. Generated automatically as the last step of every offline build (`PmtilesBuilder.extract_hiking_trails`, best-effort — a failure here doesn't fail the whole build) into `data/hiking_trails.geojson`, served at `GET /tiles/hiking_trails.geojson` only once the overlay's actually toggled on. Offline click uses a local `queryRenderedFeatures` hit-test instead of Overpass — instant, no network. See docs/development.adoc
*** [ ] File size for a many-country selection could get large (Slovakia alone: ~3000 routes) — coordinates are rounded to 5dp but there's no real tiling/simplification yet; revisit if a big selection makes the browser fetch noticeably slow
* [x] Overpass/OSM scripts to extract POIs (drinking water, dump stations, LPG, public toilets, laundromats, shops, springs) — `rake overpass:pois`, see docs/development.adoc; writes a `.geojson` to drop in `data/gpx/` or upload. Also runnable from inside the app itself now, see the import plugins entry below (`plugins/overpass-pois` wraps this same query). Each category now carries a real `POI_ICONS` glyph and color (not p4n2gpx's plain red marker) plus a popup description built from opening_hours/operator/website tags * [x] Overpass/OSM scripts to extract POIs (drinking water, dump stations, LPG, public toilets, laundromats, shops, springs) — `rake overpass:pois`, see docs/development.adoc; writes a `.geojson` to drop in `data/gpx/` or upload. Also runnable from inside the app itself now, see the import plugins entry below (`plugins/overpass-pois` wraps this same query). Each category now carries a real `POI_ICONS` glyph and color (not p4n2gpx's plain red marker) plus a popup description built from opening_hours/operator/website tags
* [x] Import plugins: a pluggable way to add more data sources (Park4Night, iOverlander, Freecampsites, ...) without their scrapers necessarily living in this (public) repo — sidebar icon (logged in only) lists whatever's under `plugins/`, "Run for this area" uses the current map viewport as the bbox, no separate area-picker needed. Contract: a `plugin.json` manifest + a script that reads `--south`/`--west`/`--north`/`--east` and prints a GeoJSON `FeatureCollection` to stdout (`lib/maptool/plugins.rb`, `GET /api/plugins`, `POST /api/plugins/:name/run`) — output saved the same way an upload would be. `plugins/overpass-pois` ships as the reference example; a private/ToS-sensitive scraper just drops its own directory in locally and is never `git add`ed * [x] Import plugins: a pluggable way to add more data sources (Park4Night, iOverlander, Freecampsites, ...) without their scrapers necessarily living in this (public) repo — sidebar icon (logged in only) lists whatever's under `plugins/`, "Run for this area" uses the current map viewport as the bbox, no separate area-picker needed. Contract: a `plugin.json` manifest + a script that reads `--south`/`--west`/`--north`/`--east` and prints a GeoJSON `FeatureCollection` to stdout (`lib/maptool/plugins.rb`, `GET /api/plugins`, `POST /api/plugins/:name/run`) — output saved the same way an upload would be. `plugins/overpass-pois` ships as the reference example; a private/ToS-sensitive scraper just drops its own directory in locally and is never `git add`ed
** [x] Park4Night — private plugin (`data/plugins/p4n2gpx/`, outside this repo per the "ToS-sensitive scraper" note above); iOverlander/Freecampsites still open, same story ** [x] Park4Night — private plugin (`data/plugins/p4n2gpx/`, outside this repo per the "ToS-sensitive scraper" note above); iOverlander/Freecampsites still open, same story
+25
View File
@@ -62,6 +62,7 @@ module Maptool
OFFLINE_PMTILES_PATH = File.join(DATA_DIR, "offline.pmtiles") OFFLINE_PMTILES_PATH = File.join(DATA_DIR, "offline.pmtiles")
TERRAIN_PMTILES_PATH = File.join(DATA_DIR, "terrain.pmtiles") TERRAIN_PMTILES_PATH = File.join(DATA_DIR, "terrain.pmtiles")
HIKING_TRAILS_PATH = PmtilesBuilder.hiking_trails_path(DATA_DIR)
GPX_STORE = GpxStore.new(File.join(DATA_DIR, "gpx")) GPX_STORE = GpxStore.new(File.join(DATA_DIR, "gpx"))
LIVETRACK_STORE = LiveTrackStore.new(File.join(DATA_DIR, "livetrack.json")) LIVETRACK_STORE = LiveTrackStore.new(File.join(DATA_DIR, "livetrack.json"))
@@ -356,6 +357,30 @@ module Maptool
json ok: true json ok: true
end end
# Generated as a step inside POST /api/offline-tiles/build (see
# PmtilesBuilder.extract_hiking_trails) rather than its own separate
# build — it's derived from the exact same per-country .osm.pbf cache
# offline.pmtiles itself already downloads, so there's nothing of its
# own to trigger or delete; DELETE /api/offline-tiles removes this
# file too. No `meta` argument here (unlike offline/terrain above) —
# parsing the whole GeoJSON just to report a feature count would cost
# real time on every page load's status check for what's a multi-MB
# file across several countries; size/mtime alone is enough to know
# whether it's there.
get "/api/hiking-trails" do
json pmtiles_file_status(HIKING_TRAILS_PATH, nil)
end
# Fetched client-side only when the Hiking trails overlay is actually
# toggled on (public/js/hiking_trails.js), not on page load — same
# "on demand" story as the online raster tiles it replaces once this
# is available.
get "/tiles/hiking_trails.geojson" do
halt 404 unless File.exist?(HIKING_TRAILS_PATH)
send_file HIKING_TRAILS_PATH, type: "application/geo+json"
end
# Pinged once per page load (public/js/map.js) — url is a same-origin # Pinged once per page load (public/js/map.js) — url is a same-origin
# path (not kiwix-serve's own address, which the browser can't reach # path (not kiwix-serve's own address, which the browser can't reach
# directly behind the reverse proxy in front of this app), proxied by # directly behind the reverse proxy in front of this app), proxied by
+44
View File
@@ -101,6 +101,50 @@ source uses — `public/js/dem.js`'s own point-elevation lookups
(`shadow.js`'s terrain-shadow overlay) read the actual decoded tile (`shadow.js`'s terrain-shadow overlay) read the actual decoded tile
size rather than assuming one, so this isn't a problem either way. size rather than assuming one, so this isn't a problem either way.
== Offline hiking trails
The sidebar's "Hiking trails" overlay works online out of the box
(waymarkedtrails.org raster tiles, `public/js/hiking_trails.js`) —
clicking a trail fires a one-off Overpass query for its name/colour/
network/distance. For it to work offline too, `data/hiking_trails.geojson`
is generated automatically as the last step of every `offline.pmtiles`
build (`Maptool::HikingTrails`, `lib/maptool/hiking_trails.rb`), straight
from the *same* per-country `.osm.pbf` files already downloaded for
`offline.pmtiles` itself — no separate download, no third-party service:
[source,shell]
----
osmium tags-filter country.osm.pbf r/route=hiking -o hiking.osm.pbf
osmium add-locations-to-ways hiking.osm.pbf -o hiking-located.osm.pbf
osmium cat hiking-located.osm.pbf -f opl,locations_on_ways=true,add_metadata=false -o hiking.opl
----
`add-locations-to-ways` matters: a plain Geofabrik PBF's ways only carry
bare node *references*, not coordinates, so skipping this step silently
produces ways with empty x/y in the OPL output and every route gets
dropped for having no usable geometry — not an error, just wrong output
(verified against a real extract before this was caught). No general OSM
tool stitches a route relation's member ways into one line for you
(routes are a semantic grouping, not a geometry type), so
`HikingTrails.parse_opl` does that walk itself — the same thing
waymarkedtrails.org and this app's own online Overpass click handler
already do.
Best-effort: a hiking-trails extraction problem on one country's PBF
logs a warning and is skipped rather than failing the whole
`offline.pmtiles` build over what's a bonus overlay, not the core map
(`PmtilesBuilder.extract_hiking_trails`). Coordinates are rounded to 5
decimal places (~1m, plenty for a rendered line) to keep the combined
file smaller — for a many-country selection it can still be a sizeable
GeoJSON (Slovakia alone: ~3000 routes, ~30MB unrounded), fetched
client-side only once the overlay is actually toggled on, not on page
load. `rake hiking:trails[pbf,out]` runs the extraction by hand against
one `.osm.pbf` for testing.
`DELETE /api/offline-tiles` removes this file along with `offline.pmtiles`
and the Geofabrik cache — there's no separate delete/rebuild route for it,
since it's not an independent build.
== In-app regeneration == In-app regeneration
A logged-in user can also generate both files above from the app A logged-in user can also generate both files above from the app
+45 -8
View File
@@ -7,18 +7,26 @@ module Maptool
# Prototype: extracts route=hiking relations (name/colour/network/operator, # Prototype: extracts route=hiking relations (name/colour/network/operator,
# full member-way geometry) straight out of a Geofabrik .osm.pbf — the same # full member-way geometry) straight out of a Geofabrik .osm.pbf — the same
# file PmtilesBuilder already downloads per country for offline.pmtiles, so # file PmtilesBuilder already downloads per country for offline.pmtiles, so
# this needs no separate download or third-party service. Two osmium # this needs no separate download or third-party service. Three osmium
# subprocess steps do the heavy lifting (osmium is already a server # subprocess steps do the heavy lifting (osmium is already a server
# dependency, see PmtilesBuilder/docs/development.adoc): # dependency, see PmtilesBuilder/docs/development.adoc):
# #
# 1. `osmium tags-filter FILE r/route=hiking -o hiking.osm.pbf` — pulls # 1. `osmium tags-filter FILE r/route=hiking -o hiking.osm.pbf` — pulls
# out just the hiking relations *and* (by default, no -R) every way/ # out just the hiking relations *and* (by default, no -R) every way/
# node they reference, so the output is small and self-contained. # node they reference, so the output is small and self-contained.
# 2. `osmium cat hiking.osm.pbf -f opl,locations_on_ways=true,add_metadata=false` # 2. `osmium add-locations-to-ways` — a plain Geofabrik PBF's ways only
# carry bare node *references* (ids), not coordinates; this resolves
# each way's node ids against the node objects in the same file and
# rewrites the way with real coordinates attached. Verified against a
# real extract (Macedonia) that skipping this step silently produces
# ways with empty x/y (`n123xy`, no numbers) in the OPL output below —
# not an error, just wrong output, so easy to miss without checking.
# 3. `osmium cat hiking.osm.pbf -f opl,locations_on_ways=true,add_metadata=false`
# — OPL is a plain-text, one-object-per-line format (see # — OPL is a plain-text, one-object-per-line format (see
# https://osmcode.org/opl-file-format/); locations_on_ways embeds each # https://osmcode.org/opl-file-format/); locations_on_ways (now that
# way's node coordinates directly in its own line, so no separate # step 2 actually populated them) embeds each way's node coordinates
# node-id -> [lon,lat] table needs building, just two passes over text. # directly in its own line, so no separate node-id -> [lon,lat] table
# needs building, just two passes over text.
# #
# No general OSM tool stitches a route relation's member ways into one # No general OSM tool stitches a route relation's member ways into one
# line for you (routes are a semantic grouping, not a geometry type) — # line for you (routes are a semantic grouping, not a geometry type) —
@@ -45,13 +53,32 @@ module Maptool
filtered = File.join(dir, "hiking.osm.pbf") filtered = File.join(dir, "hiking.osm.pbf")
run!(osmium_bin, "tags-filter", pbf_path, "r/route=hiking", "-o", filtered) run!(osmium_bin, "tags-filter", pbf_path, "r/route=hiking", "-o", filtered)
located = File.join(dir, "hiking-located.osm.pbf")
run!(osmium_bin, "add-locations-to-ways", filtered, "-o", located)
opl_path = File.join(dir, "hiking.opl") opl_path = File.join(dir, "hiking.opl")
run!(osmium_bin, "cat", filtered, "-f", "opl,locations_on_ways=true,add_metadata=false", "-o", opl_path) run!(osmium_bin, "cat", located, "-f", "opl,locations_on_ways=true,add_metadata=false", "-o", opl_path)
parse_opl(File.read(opl_path)) parse_opl(File.read(opl_path))
end end
end end
# Runs extract() over one .osm.pbf per selected country and combines the
# results into a single FeatureCollection — a route relation crossing a
# border (an E-path, say) shows up once per country whose extract it
# was pulled from, deduplicated here by OSM relation id (properties.id)
# rather than left to render as overlapping duplicate lines.
# extract: same injectable-seam reasoning as PmtilesBuilder's own
# download_extract:/fetch_manifest: — lets a test swap out the real
# (osmium-shelling-out) implementation for a canned one.
def self.extract_merged(pbf_paths, extract: method(:extract))
seen = {}
pbf_paths.each do |pbf_path|
extract.call(pbf_path)["features"].each { |feature| seen[feature["properties"]["id"]] ||= feature }
end
{ "type" => "FeatureCollection", "features" => seen.values }
end
# Pure — no subprocess — so it's cheaply testable against a canned OPL # Pure — no subprocess — so it's cheaply testable against a canned OPL
# fixture, same "keep the real logic testable without the real tool # fixture, same "keep the real logic testable without the real tool
# installed" reasoning OverpassPois.to_geojson already gives for staying # installed" reasoning OverpassPois.to_geojson already gives for staying
@@ -70,7 +97,7 @@ module Maptool
ways[head[1..].to_i] = way_coordinates(fields) ways[head[1..].to_i] = way_coordinates(fields)
when "r" when "r"
tags = tags_field(fields) tags = tags_field(fields)
relations << { tags: tags, members: members_field(fields) } if tags["route"] == "hiking" relations << { id: head[1..].to_i, tags: tags, members: members_field(fields) } if tags["route"] == "hiking"
end end
end end
@@ -88,6 +115,7 @@ module Maptool
distance = lines.sum { |coords| TrackStats.compute(coords)["distance_meters"] } distance = lines.sum { |coords| TrackStats.compute(coords)["distance_meters"] }
tags = relation[:tags] tags = relation[:tags]
properties = { properties = {
"id" => relation[:id],
"name" => tags["name"] || tags["description"], "name" => tags["name"] || tags["description"],
"distance_meters" => distance.round(1), "distance_meters" => distance.round(1),
} }
@@ -96,11 +124,20 @@ module Maptool
{ {
"type" => "Feature", "type" => "Feature",
"properties" => properties.compact, "properties" => properties.compact,
"geometry" => { "type" => "MultiLineString", "coordinates" => lines }, # Rounded to 5 decimal places (~1m) — full OSM float precision
# (~1cm) is pointless for a rendered line and roughly doubles the
# size of every coordinate pair once this many-country GeoJSON is
# actually served to a browser.
"geometry" => { "type" => "MultiLineString", "coordinates" => round_coordinates(lines) },
} }
end end
private_class_method :relation_feature private_class_method :relation_feature
def self.round_coordinates(lines)
lines.map { |coords| coords.map { |lon, lat| [lon.round(5), lat.round(5)] } }
end
private_class_method :round_coordinates
def self.field(fields, letter) def self.field(fields, letter)
fields.find { |f| f.start_with?(letter) } fields.find { |f| f.start_with?(letter) }
end end
+27
View File
@@ -5,6 +5,7 @@ require "json"
require "tmpdir" require "tmpdir"
require "fileutils" require "fileutils"
require "etc" require "etc"
require_relative "hiking_trails"
module Maptool module Maptool
# Regenerates data/offline.pmtiles and data/terrain.pmtiles for the # Regenerates data/offline.pmtiles and data/terrain.pmtiles for the
@@ -289,12 +290,37 @@ module Maptool
run_planetiler!("--osm-path=#{osm_path}", "--output=#{tmp_path}", "--download", "--threads=#{Etc.nprocessors}") run_planetiler!("--osm-path=#{osm_path}", "--output=#{tmp_path}", "--download", "--threads=#{Etc.nprocessors}")
File.rename(tmp_path, File.join(data_dir, "offline.pmtiles")) File.rename(tmp_path, File.join(data_dir, "offline.pmtiles"))
File.write(offline_meta_path(data_dir), JSON.generate(sources: sources.map { |s| s.slice("code", "slug", "last_modified") })) File.write(offline_meta_path(data_dir), JSON.generate(sources: sources.map { |s| s.slice("code", "slug", "last_modified") }))
report_step.call("extracting hiking trails")
extract_hiking_trails(sources, data_dir)
ensure ensure
FileUtils.remove_entry(scratch_dir) if scratch_dir && Dir.exist?(scratch_dir) FileUtils.remove_entry(scratch_dir) if scratch_dir && Dir.exist?(scratch_dir)
File.delete(tmp_path) if tmp_path && File.exist?(tmp_path) File.delete(tmp_path) if tmp_path && File.exist?(tmp_path)
end end
private_class_method :extract_offline private_class_method :extract_offline
# Runs over the same per-country .osm.pbf files sources above already
# downloaded/cached for offline.pmtiles itself — no separate download.
# Best-effort: a hiking-trails extraction problem (a malformed PBF
# region, osmium hiccup on one particular file) shouldn't fail the
# whole offline.pmtiles build over what's a bonus overlay, not the
# core map — logged into the sidecar meta's own "error" field instead,
# so it's visible without taking build_offline itself down.
def self.extract_hiking_trails(sources, data_dir)
geojson = HikingTrails.extract_merged(sources.map { |s| s["path"] })
tmp_path = "#{hiking_trails_path(data_dir)}.tmp"
File.write(tmp_path, JSON.generate(geojson))
File.rename(tmp_path, hiking_trails_path(data_dir))
rescue HikingTrails::ExtractError => e
File.delete(tmp_path) if tmp_path && File.exist?(tmp_path)
warn "hiking trails extraction failed, skipping: #{e.message}"
end
private_class_method :extract_hiking_trails
def self.hiking_trails_path(data_dir)
File.join(data_dir, "hiking_trails.geojson")
end
# Renders a report_step string like "fetching PL (3/4) — done: SK, CZ — # Renders a report_step string like "fetching PL (3/4) — done: SK, CZ —
# pending: HU" — so GET .../build-status shows not just "it's running" # pending: HU" — so GET .../build-status shows not just "it's running"
# but which of the current build's sources is in flight and which are # but which of the current build's sources is in flight and which are
@@ -442,6 +468,7 @@ module Maptool
File.delete(File.join(data_dir, "offline.pmtiles")) if File.exist?(File.join(data_dir, "offline.pmtiles")) File.delete(File.join(data_dir, "offline.pmtiles")) if File.exist?(File.join(data_dir, "offline.pmtiles"))
File.delete(offline_meta_path(data_dir)) if File.exist?(offline_meta_path(data_dir)) File.delete(offline_meta_path(data_dir)) if File.exist?(offline_meta_path(data_dir))
File.delete(hiking_trails_path(data_dir)) if File.exist?(hiking_trails_path(data_dir))
cache_dir = geofabrik_cache_dir(data_dir) cache_dir = geofabrik_cache_dir(data_dir)
FileUtils.remove_entry(cache_dir) if Dir.exist?(cache_dir) FileUtils.remove_entry(cache_dir) if Dir.exist?(cache_dir)
end end
+129 -19
View File
@@ -34,21 +34,45 @@ const SAFE_COLOR = /^(#[0-9a-fA-F]{3,8}|[a-zA-Z]+)$/;
// Pre-rendered raster tiles, not vector — waymarkedtrails.org already // Pre-rendered raster tiles, not vector — waymarkedtrails.org already
// colors each route by its OSM `colour` tag (the KST/KČT-style paint-blaze // colors each route by its OSM `colour` tag (the KST/KČT-style paint-blaze
// colors used in Slovakia/Czechia), falling back to plain red where a route // colors used in Slovakia/Czechia), falling back to plain red where a route
// has none, so there's no styling to do here ourselves. Online only, same // has none, so there's no styling to do here ourselves. Used when there's
// "quietly renders nothing without a network" story as the Wikipedia/Radar/ // no local offline extraction to draw from instead (see _addOfflineLayer
// Contour lines overlays — an offline copy (baked into offline.pmtiles) is // below) — same "quietly renders nothing without a network" story as the
// a TODO for once there's a route planner to pair it with, see TODO.adoc. // Wikipedia/Radar/Contour lines overlays.
const TILE_URL = "https://tile.waymarkedtrails.org/hiking/{z}/{x}/{y}.png"; const TILE_URL = "https://tile.waymarkedtrails.org/hiking/{z}/{x}/{y}.png";
const ATTRIBUTION = const ATTRIBUTION =
'Trails &copy; <a href="https://waymarkedtrails.org" target="_blank">Waymarked Trails</a> (CC-BY-SA), map data &copy; OpenStreetMap contributors'; 'Trails &copy; <a href="https://waymarkedtrails.org" target="_blank">Waymarked Trails</a> (CC-BY-SA), map data &copy; OpenStreetMap contributors';
const OPACITY = 0.85; const OPACITY = 0.85;
// Offline: a real vector line layer, GeoJSON generated at build time by
// Maptool::HikingTrails straight from the same Geofabrik .osm.pbf
// offline.pmtiles itself is built from (see lib/maptool/hiking_trails.rb,
// PmtilesBuilder.extract_offline) — no network needed at all, and no
// waiting on Overpass for click details either (queried locally via
// queryRenderedFeatures instead, see _handleClick below).
const OFFLINE_GEOJSON_URL = "/tiles/hiking_trails.geojson";
// waymarkedtrails.org's own fallback for an untagged route.
const DEFAULT_TRAIL_COLOR = "#c1272d";
// Real-world `colour` values seen in production data (see the 2026-09-13
// extraction against SK.osm.pbf) — anything outside this set (typos like
// "yelllow", multi-value combos like "blue, yellow", stray hex codes) falls
// back to DEFAULT_TRAIL_COLOR rather than being trusted as a CSS value
// straight from OSM tag text.
const KNOWN_TRAIL_COLORS = new Set(["red", "yellow", "blue", "green", "orange", "purple", "brown", "black", "white", "grey", "gray", "darkblue"]);
function normalizedTrailColor(raw) {
if (!raw) return DEFAULT_TRAIL_COLOR;
const trimmed = raw.trim();
if (KNOWN_TRAIL_COLORS.has(trimmed.toLowerCase())) return trimmed.toLowerCase();
return /^#[0-9a-fA-F]{3,8}$/.test(trimmed) ? trimmed : DEFAULT_TRAIL_COLOR;
}
// A plain on/off Overlays item, same shape as ContourControl — one static // A plain on/off Overlays item, same shape as ContourControl — one static
// raster overlay, no scrub/forecast controls to expand into. // raster overlay, no scrub/forecast controls to expand into.
export class HikingTrailsControl { export class HikingTrailsControl {
constructor(map) { constructor(map) {
this._map = map; this._map = map;
this._active = false; this._active = false;
this._offlineAvailable = false;
this._clickHandler = (event) => this._handleClick(event); this._clickHandler = (event) => this._handleClick(event);
// setStyle() (switching Online/Offline) tears down all custom sources/ // setStyle() (switching Online/Offline) tears down all custom sources/
@@ -59,6 +83,14 @@ export class HikingTrailsControl {
}); });
} }
// Called once map.js's GET /api/hiking-trails check resolves — same
// "prefer the local file whenever it exists" story as
// OverlaysControl#setHillshadeOfflineAvailable, independent of whether
// the Online or Offline *base map* happens to be selected right now.
setOfflineAvailable(available) {
this._offlineAvailable = available;
}
mount(li) { mount(li) {
li.innerHTML = ` li.innerHTML = `
<label> <label>
@@ -82,12 +114,36 @@ export class HikingTrailsControl {
} }
} }
// The raster tiles above have no queryable features (they're pixels), so _handleClick(event) {
if (this._offlineAvailable) {
this._handleOfflineClick(event);
} else {
this._handleOnlineClick(event);
}
}
// The local vector layer has real features to hit-test directly — no
// network round-trip, so this resolves instantly. A genuine miss (empty
// result) shows no popup at all rather than a "not found" message, same
// as clicking empty space anywhere else on the map.
_handleOfflineClick(event) {
const features = this._map.queryRenderedFeatures(event.point, { layers: [LAYER_ID] });
if (features.length === 0) return;
const seen = new Set();
const html = features
.filter((feature) => !seen.has(feature.properties.id) && seen.add(feature.properties.id))
.map((feature) => routeHtml(feature.properties))
.join("<hr>");
new Popup({ closeButton: true, maxWidth: "260px" }).setLngLat(event.lngLat).setHTML(html).addTo(this._map);
}
// The raster tiles have no queryable features (they're pixels), so
// "what trail is this" is answered with a one-off Overpass query instead // "what trail is this" is answered with a one-off Overpass query instead
// of a MapLibre feature hit-test — an `around` search rather than an // of a MapLibre feature hit-test — an `around` search rather than an
// exact point match, since a click rarely lands exactly on the (already // exact point match, since a click rarely lands exactly on the (already
// pixel-thick) rendered line. // pixel-thick) rendered line.
async _handleClick(event) { async _handleOnlineClick(event) {
const { lng, lat } = event.lngLat; const { lng, lat } = event.lngLat;
const popup = new Popup({ closeButton: true, maxWidth: "260px" }) const popup = new Popup({ closeButton: true, maxWidth: "260px" })
.setLngLat(event.lngLat) .setLngLat(event.lngLat)
@@ -104,14 +160,49 @@ export class HikingTrailsControl {
popup.setHTML( popup.setHTML(
relations.length > 0 relations.length > 0
? relations.map(routeHtml).join("<hr>") ? relations.map((relation) => routeHtml(overpassRouteInfo(relation))).join("<hr>")
: `<p class="gpx-popup__category">No hiking trail found here.</p>`, : `<p class="gpx-popup__category">No hiking trail found here.</p>`,
); );
} }
_addLayer() { async _addLayer() {
if (this._map.getSource(SOURCE_ID)) return; if (this._map.getSource(SOURCE_ID)) return;
if (this._offlineAvailable) await this._addOfflineLayer();
else this._addOnlineLayer();
}
async _addOfflineLayer() {
let geojson;
try {
geojson = await (await fetch(OFFLINE_GEOJSON_URL)).json();
} catch {
return; // degrade quietly, same story as every other optional layer here
}
if (this._map.getSource(SOURCE_ID)) return; // toggled off and back on again while this was fetching
// Normalized once here (rather than a MapLibre expression in the paint
// property below) so an untrustworthy `colour` tag value never reaches
// a style property directly — same reasoning gpx.js's own
// feature.properties.color ||= TRACK_COLOR normalization gives.
for (const feature of geojson.features) {
feature.properties.color = normalizedTrailColor(feature.properties.colour);
}
this._map.addSource(SOURCE_ID, { type: "geojson", data: geojson });
this._map.addLayer(
{
id: LAYER_ID,
type: "line",
source: SOURCE_ID,
layout: { "line-cap": "round", "line-join": "round" },
paint: { "line-color": ["get", "color"], "line-width": 2 },
},
firstSymbolLayerId(this._map),
);
}
_addOnlineLayer() {
this._map.addSource(SOURCE_ID, { this._map.addSource(SOURCE_ID, {
type: "raster", type: "raster",
tiles: [TILE_URL], tiles: [TILE_URL],
@@ -154,23 +245,42 @@ async function fetchNearbyRoutes(lng, lat) {
// OSM doesn't tag a route's length directly — summed from each member // OSM doesn't tag a route's length directly — summed from each member
// way's own geometry instead, same haversine-sum measure.js's own Measure // way's own geometry instead, same haversine-sum measure.js's own Measure
// tool uses for a hand-drawn line. Shared endpoints between adjacent ways // tool uses for a hand-drawn line. Shared endpoints between adjacent ways
// get walked twice, same as how a real hike over those ways would. // get walked twice, same as how a real hike over those ways would. Mirrors
// Maptool::HikingTrails.relation_feature's own distance calc server-side
// (lib/maptool/hiking_trails.rb) — this is the online, no-server-file path.
function routeDistance(relation) { function routeDistance(relation) {
return (relation.members || []) return (relation.members || [])
.filter((member) => member.type === "way" && member.geometry) .filter((member) => member.type === "way" && member.geometry)
.reduce((total, member) => total + lineLength(member.geometry.map((point) => [point.lon, point.lat])), 0); .reduce((total, member) => total + lineLength(member.geometry.map((point) => [point.lon, point.lat])), 0);
} }
function routeHtml(relation) { // Adapts a raw Overpass relation into the same shape offline GeoJSON
// features' properties already carry (lib/maptool/hiking_trails.rb's own
// ROUTE_TAGS + distance_meters + id) — one popup builder below serves
// both instead of two near-identical ones.
function overpassRouteInfo(relation) {
const tags = relation.tags || {}; const tags = relation.tags || {};
const name = tags.name || tags.description || "Hiking trail"; return {
const swatch = SAFE_COLOR.test(tags.colour || "") id: relation.id,
? `<span class="hiking-trail-popup__swatch" style="background:${tags.colour}"></span>` name: tags.name,
description: tags.description,
colour: tags.colour,
network: tags.network,
operator: tags.operator,
ref: tags.ref,
distance_meters: routeDistance(relation),
};
}
function routeHtml(info) {
const name = info.name || info.description || "Hiking trail";
const swatch = SAFE_COLOR.test(info.colour || "")
? `<span class="hiking-trail-popup__swatch" style="background:${info.colour}"></span>`
: ""; : "";
const meta = [formatDistance(routeDistance(relation))]; const meta = [formatDistance(info.distance_meters || 0)];
if (tags.ref) meta.push(`#${tags.ref}`); if (info.ref) meta.push(`#${info.ref}`);
if (tags.network) meta.push(tags.network); if (info.network) meta.push(info.network);
if (tags.operator) meta.push(escapeHtml(tags.operator)); if (info.operator) meta.push(escapeHtml(info.operator));
const parts = [ const parts = [
`<div class="gpx-popup__header"><strong>${escapeHtml(name)}</strong>${swatch}</div>`, `<div class="gpx-popup__header"><strong>${escapeHtml(name)}</strong>${swatch}</div>`,
@@ -180,9 +290,9 @@ function routeHtml(relation) {
// shown again here as its own line if there both is a name *and* a // shown again here as its own line if there both is a name *and* a
// description to add, so it's never rendered twice for the common // description to add, so it's never rendered twice for the common
// no-name case. // no-name case.
if (tags.name && tags.description) parts.push(`<p class="gpx-popup__description">${escapeHtml(tags.description)}</p>`); if (info.name && info.description) parts.push(`<p class="gpx-popup__description">${escapeHtml(info.description)}</p>`);
parts.push( parts.push(
`<p class="gpx-popup__link"><a href="https://www.openstreetmap.org/relation/${relation.id}" target="_blank" rel="noopener noreferrer">View on OpenStreetMap →</a></p>`, `<p class="gpx-popup__link"><a href="https://www.openstreetmap.org/relation/${info.id}" target="_blank" rel="noopener noreferrer">View on OpenStreetMap →</a></p>`,
); );
return parts.join(""); return parts.join("");
} }
+5
View File
@@ -607,6 +607,11 @@ fetch("/api/terrain-tiles")
}) })
.catch(() => {}); .catch(() => {});
fetch("/api/hiking-trails")
.then((response) => response.json())
.then(({ available }) => overlaysControl.setHikingTrailsOfflineAvailable(available))
.catch(() => {});
fetch("/api/kiwix") fetch("/api/kiwix")
.then((response) => response.json()) .then((response) => response.json())
.then(({ available }) => overlaysControl.setWikiAvailable(available)) .then(({ available }) => overlaysControl.setWikiAvailable(available))
+4
View File
@@ -245,6 +245,10 @@ export class OverlaysControl {
this._hillshade.setOfflineAvailable(available); this._hillshade.setOfflineAvailable(available);
} }
setHikingTrailsOfflineAvailable(available) {
this._hikingTrails.setOfflineAvailable(available);
}
// Called by map.js's PitchToggleControl whenever the 3D pitch toggle // Called by map.js's PitchToggleControl whenever the 3D pitch toggle
// changes, so raised terrain follows the camera tilt rather than needing // changes, so raised terrain follows the camera tilt rather than needing
// the "Terrain shading" checkbox re-clicked to notice. // the "Terrain shading" checkbox re-clicked to notice.
+9
View File
@@ -39,6 +39,15 @@ class AppTest < Minitest::Test
assert_equal({ "available" => false }, json_body(response)) assert_equal({ "available" => false }, json_body(response))
end end
def test_hiking_trails_reports_unavailable_when_no_geojson_present
response = request.get("/api/hiking-trails")
assert_equal({ "available" => false }, json_body(response))
end
def test_hiking_trails_file_404s_when_no_geojson_present
assert_equal 404, request.get("/tiles/hiking_trails.geojson").status
end
def test_kiwix_reports_unavailable_when_no_url_configured def test_kiwix_reports_unavailable_when_no_url_configured
response = request.get("/api/kiwix") response = request.get("/api/kiwix")
assert_equal({ "available" => false, "url" => "" }, json_body(response)) assert_equal({ "available" => false, "url" => "" }, json_body(response))
+37
View File
@@ -89,6 +89,43 @@ class HikingTrailsTest < Minitest::Test
assert_empty Maptool::HikingTrails.parse_opl(opl)["features"] assert_empty Maptool::HikingTrails.parse_opl(opl)["features"]
end end
def test_parse_opl_carries_the_osm_relation_id_for_cross_country_dedup
opl = <<~OPL
w1 Thighway=path Nn10x20.087y49.179,n11x20.088y49.18
r1246205 Troute=hiking,type=route,name=Trail Mw1@
OPL
feature = Maptool::HikingTrails.parse_opl(opl)["features"].first
assert_equal 1246205, feature["properties"]["id"]
end
def test_parse_opl_rounds_coordinates_to_5_decimal_places
opl = <<~OPL
w1 Thighway=path Nn10x20.0876543210y49.1791234567,n11x20.0880000001y49.1800000009
r1 Troute=hiking,type=route,name=Trail Mw1@
OPL
coords = Maptool::HikingTrails.parse_opl(opl)["features"].first["geometry"]["coordinates"]
assert_equal [[[20.08765, 49.17912], [20.088, 49.18]]], coords
end
def test_extract_merged_deduplicates_by_relation_id_across_files
feature = ->(id) {
{ "type" => "Feature", "properties" => { "id" => id, "name" => "Trail #{id}" }, "geometry" => { "type" => "MultiLineString", "coordinates" => [] } }
}
# Relation 1 shows up in both "country" files (a cross-border trail);
# relation 2 only in the second.
per_file = { "at.osm.pbf" => [feature.call(1)], "cz.osm.pbf" => [feature.call(1), feature.call(2)] }
extract = ->(pbf_path) { { "type" => "FeatureCollection", "features" => per_file.fetch(pbf_path) } }
merged = Maptool::HikingTrails.extract_merged(%w[at.osm.pbf cz.osm.pbf], extract: extract)
ids = merged["features"].map { |f| f["properties"]["id"] }.sort
assert_equal [1, 2], ids
end
def test_parse_opl_ignores_a_dangling_member_reference_but_keeps_the_rest def test_parse_opl_ignores_a_dangling_member_reference_but_keeps_the_rest
opl = <<~OPL opl = <<~OPL
w1 Thighway=path Nn10x20.087y49.179,n11x20.088y49.18 w1 Thighway=path Nn10x20.087y49.179,n11x20.088y49.18
+59 -2
View File
@@ -76,6 +76,27 @@ class PmtilesBuilderTest < Minitest::Test
SH SH
File.chmod(0o755, @osmium_ok) File.chmod(0o755, @osmium_ok)
# Same shape as osmium-ok, but fails on tags-filter specifically (the
# first subcommand HikingTrails.extract runs) while still succeeding
# on merge — lets a single-country build (which never calls merge)
# exercise "hiking-trails extraction failed" without also breaking
# offline.pmtiles' own osmium use in a multi-country test.
@osmium_fail_tags_filter = File.join(@bin_dir, "osmium-fail-tags-filter")
File.write(@osmium_fail_tags_filter, <<~SH)
#!/bin/sh
echo "osmium $1" >> "$PMTILES_LOG"
if [ "$1" = "tags-filter" ]; then
echo "boom" >&2
exit 1
fi
prev=""
for arg; do
if [ "$prev" = "-o" ]; then echo fake > "$arg"; fi
prev="$arg"
done
SH
File.chmod(0o755, @osmium_fail_tags_filter)
# Stands in for `java -jar planetiler.jar --osm-path=... --output=OUT` # Stands in for `java -jar planetiler.jar --osm-path=... --output=OUT`
# — PLANETILER_JAR itself is irrelevant here (this fake ignores -jar's # — PLANETILER_JAR itself is irrelevant here (this fake ignores -jar's
# argument entirely), only JAVA_BIN needs pointing at it. # argument entirely), only JAVA_BIN needs pointing at it.
@@ -188,13 +209,42 @@ class PmtilesBuilderTest < Minitest::Test
assert_path_exists File.join(@data_dir, "offline.pmtiles") assert_path_exists File.join(@data_dir, "offline.pmtiles")
refute_path_exists File.join(@data_dir, "offline.tmp.pmtiles") refute_path_exists File.join(@data_dir, "offline.tmp.pmtiles")
assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state] assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state]
assert_equal ["planetiler --osm-path"], log_lines assert_equal ["planetiler --osm-path", "osmium tags-filter", "osmium add-locations-to-ways", "osmium cat"], log_lines
assert_equal( assert_equal(
{ "sources" => [{ "code" => "SK", "slug" => "europe/slovakia", "last_modified" => "Wed, 10 Sep 2026 08:00:01 GMT" }] }, { "sources" => [{ "code" => "SK", "slug" => "europe/slovakia", "last_modified" => "Wed, 10 Sep 2026 08:00:01 GMT" }] },
Maptool::PmtilesBuilder.offline_meta(@data_dir), Maptool::PmtilesBuilder.offline_meta(@data_dir),
) )
end end
def test_build_offline_also_writes_a_hiking_trails_geojson
select_regions("SK")
thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, download_extract: fake_download)
thread.join
# The fake osmium binary doesn't produce real OPL output, just
# confirms the file lands and is valid (empty) GeoJSON — the real
# parsing logic is covered by test_hiking_trails.rb against real OPL
# fixtures.
path = File.join(@data_dir, "hiking_trails.geojson")
assert_path_exists path
assert_equal({ "type" => "FeatureCollection", "features" => [] }, JSON.parse(File.read(path)))
end
# The actual reasoning: hiking trails are a bonus overlay bolted onto the
# existing offline.pmtiles build, not the core map — an osmium hiccup on
# this one step shouldn't take down an otherwise-successful build.
def test_build_offline_still_succeeds_when_hiking_trails_extraction_fails
select_regions("SK")
ENV["OSMIUM_BIN"] = @osmium_fail_tags_filter
thread = Maptool::PmtilesBuilder.build_offline(data_dir: @data_dir, download_extract: fake_download)
thread.join
assert_path_exists File.join(@data_dir, "offline.pmtiles")
assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state]
refute_path_exists File.join(@data_dir, "hiking_trails.geojson")
refute_path_exists "#{File.join(@data_dir, "hiking_trails.geojson")}.tmp"
end
# Two countries selected — osmium merge should run before planetiler, # Two countries selected — osmium merge should run before planetiler,
# each downloaded source recorded in the sidecar meta. # each downloaded source recorded in the sidecar meta.
def test_build_offline_merges_multiple_selected_countries def test_build_offline_merges_multiple_selected_countries
@@ -203,7 +253,14 @@ class PmtilesBuilderTest < Minitest::Test
thread.join thread.join
assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state] assert_equal "done", Maptool::PmtilesBuilder.offline_status[:state]
assert_equal ["osmium merge", "planetiler --osm-path"], log_lines assert_equal(
[
"osmium merge", "planetiler --osm-path",
"osmium tags-filter", "osmium add-locations-to-ways", "osmium cat", # SK's own hiking-trails extraction
"osmium tags-filter", "osmium add-locations-to-ways", "osmium cat", # AT's own
],
log_lines,
)
codes = Maptool::PmtilesBuilder.offline_meta(@data_dir)["sources"].map { |s| s["code"] } codes = Maptool::PmtilesBuilder.offline_meta(@data_dir)["sources"].map { |s| s["code"] }
assert_equal %w[SK AT], codes assert_equal %w[SK AT], codes
end end