Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://gabrielmahler.org/walkability/ai/ml/2025/06/05/evaluation1",
3 "title": "Walkability Chapter 5: Evaluation (Part 1: Routing Scenarios)",
4 "link": "https://gabrielmahler.org/walkability/ai/ml/2025/06/05/evaluation1.html",
5 "updated": "2025-06-05T10:40:11",
6 "published": "2025-06-05T10:40:11",
7 "summary": "Evaluation",
8 "content": "<h1>Evaluation</h1>\n\n<p>We demonstrate that with our embedding-based routing framework, we can\ngenerate pedestrian paths that can not only align much closer to\nelaborate preferences, but also make inputting these preferences much\nsimpler, particularly in contrast to popular path-finding frameworks. To\nfully leverage our aggregated knowledge base, our experiments are\nconducted within the boundaries of the city of Cambridge, UK.</p>\n\n<p>First, through the lens of walkability, we illustrate shortcomings of\nthe path-finding frameworks (OSRM,\nValhalla, and GraphHopper) discussed in the earlier posts on a number of representative start-target\nrouting scenarios. In particular, we identify routing opportunities\nignored by these baselines that offered more pedestrian-friendly paths.</p>\n\n<p>Then, we employ the anchor-based scoring pipeline to generate “general”\nwalkability maps, and analyze their scores to contrast the outputs from\ndifferent encoder model variants. Furthermore, we conceive four\nrealistic preference objectives that address some of the specific\ndemonstrated shortcomings of the pedestrian profiles used by the\nopen-source frameworks. Separately, these aim to maximize the presence\nof 1) nature and green spaces, 2) shopping-related areas, 3)\nhistorically relevant places, and 4) public safety-inducing factors. We\nanalyze the outputs based on these objectives, both individually and\nrelative to outputs generated based on the general walkability\ncriterion.</p>\n\n<p>Finally, we use both the general walkability and preference-based scores\nto generate pedestrian paths for the same set of start-target\ndestination pairs. We use these examples to highlight our framework’s\nability to generate both walkability-focused and custom preference-based\npaths, and to do so with significantly simplified preference definitions\nthan the baseline alternatives.</p>\n\n<h2>Pedestrian Routing Scenarios</h2>\n\n<p>We define a corpus of five realistic start-target routing destination\npairs in an urban environment, selected to represent a diverse set of\narchetypal walking scenarios. Together with these pairs, we present\nrouting solutions generated by the three open-source path-finding\nframeworks (Valhalla, OSRM, and GraphHopper). We discuss these solutions\nindividually, highlighting their weaknesses, and use them to answer our\nfirst research question - what are the <strong>shortcomings of existing\npath-finding</strong> frameworks.</p>\n\n<h3>Pedestrian Routing Profiles</h3>\n\n<p>To generate the paths with the OSRM, we relied on a generic pedestrian\nprofile developed by FOSSGIS (Free and Geographical Information Systems\n2025), which is widely used in OSRM implementations. In the case of\nValhalla and GraphHopper, the path generation was based on pedestrian\nrouting profiles published directly by the frameworks’ respective\ndevelopers (Valhalla 2025; GraphHopper 2025). We show that\nthese profiles are: 1) highly complicated to configure, and 2) consider\nonly a narrow selection of strictly segment-related elements, ultimately\nmaking them hardly employable by individual users for their specific\npreferences.</p>\n\n<p>This OSRM routing profile is configured in a several-hundred-line-long\nLua file (Free and Geographical Information Systems 2025). It\nprioritizes pedestrian-designated segments and penalizes U-turns and\ntraffic lights. Furthermore, it determines which nodes directly\nassociated with the segments (such as bollards or gates) are permissible\nand which are not, and assigns exact speed changes to specific surfaces\n(such as mud or gravel).</p>\n\n<p>The profile used to generate paths with GraphHopper is defined with\nmultiple YAML and JSON files, as well as GraphHopper’s various internal\nconstants (GraphHopper 2025). Similarly to OSRM, it encodes elementary\ninformation about segments (such as surfaces or pedestrian-designated\nprofiles), which are then used to calculate the estimated travel times\nper segment. These times are then used to calculate the fastest option.</p>\n\n<p>Finally, Valhalla uses a large and complex C++ file to define its\npedestrian routing logic (Valhalla 2025). It considers a\nvariety of constraints, such as maximum distances, preferred surfaces,\nor segment types, and penalizes elements such as steps, crossings, or\nelevation. It also favors pedestrian paths and sidewalks while\ndiscouraging alleys or driveways.</p>\n\n<h3>Example 1: Supermarket-adjacent Walk</h3>\n\n<p>The first exemplary problem we employ is defined by a start destination\nclose to Cambridge’s city center, and a target in a slightly more remote\nsupermarket-adjacent area. Both points are situated in residential\nareas, and are specific by a proximity to highly walkable spaces\n(particularly parks).</p>\n\n<p><img alt=\"alt text\" src=\"https://gabrielmahler.org/assets/images/thesis/new%20images/parkside%20lidl/baselines%20parkside%20lidl.jpeg\"></p>\n\n<p>Nevertheless, none of the generated solutions takes advantage of the\nmentioned green spaces. OSRM and GraphHopper both\nfollow a similar route following a pedestrian footpath leading through a\nnearby shopping mall, and then an adjacent residential area.</p>\n\n<p>Valhalla, in contrast, sticks to sidewalks attached to busy major roads,\nresulting in a more straightforward solution. Although simple to follow,\nit does not seek to maximize the pedestrian experience. We hypothesize\nthis is driven mainly by Valhalla’s minimization of infrastructure\nfeatures (such as junctions and turns) with its penalty costing\nfunction.</p>\n\n<h3>Example 2: Long City-spanning Walk</h3>\n\n<p>The second routing example defines a problem of substantially broader\nspatial proportions, stretched between two remote destinations at\nopposite ends of Cambridge. Here too is the problem’s space is\nlargely defined by residential areas, but presents an even greater\nopportunity to leverage greenery towards maximizing walkability.</p>\n\n<p><img alt=\"alt text\" src=\"https://gabrielmahler.org/assets/images/thesis/new%20images/random%20long/baseline%20random%20long.jpeg\"></p>\n\n<p>However, the solutions generated by the open-source frameworks carry much resemblance to those\nin the previous problem. Again, OSRM and GraphHopper utilize mostly\npaths and sidewalks in residential areas, and do not pursue routing\nopportunities presented by the parks. Nonetheless, in this regard,\nValhalla seems to perform slightly better, incorporating significantly\nmore greenery into its output. However, considering its costing\nmechanisms, we attribute this (again) to Valhalla’s preference towards\navoiding complicated infrastructural features, such as junctions or\ncrossings (as is also apparent from the distinctly simple shape of its\nroute).</p>\n\n<h3>Example 3: Greenbelt Walk</h3>\n\n<p>In our third example, we bring the routing scenario closer to dense\nurban environments - between a historical university college in the city\ncenter and another, slightly more remote, college. Here, the routing\nalgorithms are provided with the option to either align their solutions\nwith the walkable but remote playing fields and parks, or leverage the\nhistorical and pedestrian-friendly city center.</p>\n\n<p><img alt=\"alt text\" src=\"https://gabrielmahler.org/assets/images/thesis/new%20images/ph%20cst%20short/ph%20cst%20baselines.jpeg\"></p>\n\n<p>The considered open-source\nframeworks mostly leverage the first option. Even though the\nOSRM-generated route delves slightly further into the historical center,\nit later joins the other two outputs in exploiting the direct but remote\nfootpaths and sidewalks outside of the urban center. This is because\nnone of these frameworks implements a mechanism with the ability to\nrecognize or reward subjectively enticing environments, such as\nCambridge’s historical center.</p>\n\n<h3>Example 4: City Center Walk</h3>\n\n<p>The fourth example in our set of routing scenarios presents a dense\nurban environment, with many opportunities for specific user\npreferences, invocable by, for example, various shopping areas,\nrestaurants, or museums.</p>\n\n<p><img alt=\"alt text\" src=\"https://gabrielmahler.org/assets/images/thesis/new%20images/ps%20senate/ph%20senate%20baselines.jpeg\"></p>\n\n<p>The open-source frameworks, however, allow for no option to input these\npreferences into their path generation. As illustrated by this routing\nscenario, the paths subsequently result in\nplain solutions, maximizing no specific objective that could be pursued\nin an environment of this kind.</p>\n\n<h3>Example 5: Suburban Stretch Walk</h3>\n\n<p>In the final routing scenario, we utilize a start-target destination\npair in a deeply residential area on the outskirts of Cambridge. We\ninclude this example to highlight the importance of subtle nuances,\nparticularly in segments’ busyness, that are relatable to important\npedestrian factors, such as the feelings of public safety.</p>\n\n<p><img alt=\"alt text\" src=\"https://gabrielmahler.org/assets/images/thesis/new%20images/residential/residential%20baselines.jpeg\"></p>\n\n<p>This scenario produces another\nshowing of divergence between the solutions generated by GraphHopper and\nOSRM, and Valhalla. Once again, we attribute this to Valhalla’s\ninclination towards spatially straightforward routes, which is reflected\nin a path that simply follows a singular road with no turns.</p>\n\n<h3>References</h3>\n\n<ul>\n <li>GraphHopper. (2025). <em>GraphHopper Routing Engine</em>.\n<a href=\"https://github.com/graphhopper/graphhopper\">https://github.com/graphhopper/graphhopper</a></li>\n <li>Free and Open Source Software for Geographical Information Systems. (2025). <em>FOSSGIS e.v.</em> <em>GitHub</em>.\n<a href=\"https://github.com/fossgis\">https://github.com/fossgis</a></li>\n <li>Valhalla. (2025). <em>Valhalla: Open Source Routing Engine for OpenStreetMap</em>. <em>GitHub Repository</em>.\n<a href=\"https://github.com/valhalla/valhalla\">https://github.com/valhalla/valhalla</a></li>\n</ul>",
9 "content_type": "html",
10 "author": {
11 "name": "",
12 "email": null,
13 "uri": null
14 },
15 "categories": [
16 "Walkability",
17 "AI/ML"
18 ],
19 "source": "https://gabrielmahler.org/feed.xml"
20}