Write new routing plugin

[taylor.smock]

Problem:
Mappers attempting to debug why a routing engine does stupid things or takes poor routes.

Solution:
Show mappers a route with route instructions as they edit live.

Goals:

1. Use production routing engines
   • ​valhalla
   • ​OsmAnd
   • ​OSRM
   • ​Organic Maps
2. Show routing instructions in JOSM for given modes (pedestrian, biking, vehicle, and possibly public transport)
3. [Stretch goal] If routing engine supports it, show weights for ways in given mode; this should allow debugging why a different route was not chosen.

---

MVP will use one of the specified routing engines and show routing instructions in JOSM.
I am currently investigating using valhalla for the initial routing engine. This will most likely require Java 22 for ​FFI support without JVM flags. Alternatively, I can use JNI or a derivative thereof. As an additional alternative, if the routing engine exposes a web interface or a command line interface, we can use those for routing support.

Possible implementation details:
Valhalla (MIT):

• Provides a C++ library (libvalhalla)
• Provides a web interface
• Provides a ​CLI

OSRM (BSD 2-clause):

• Provides a C++ library
• Provides a ​web interface

OSMAnd (GPLv3):

• Java library and bindings already written (thanks to Android)
• Has some kind of web interface (see comment:2)

Organic Maps (Apache 2):

• Java library and bindings already written (thanks to Android)

---

Most likely, I will have to write the current edit layer to pbf in order to load it into a routing engine. There may be additional steps, such as converting the pbf to the engine specific format.

---

This plugin should completely replace the routing plugin which uses custom Java code for routing.

[IreuN]

FYI OsmAnd also provides web interface at ​https://osmand.net/map/navigate/ and ​https://test.osmand.net/map/navigate/

[taylor.smock]

I probably can't send the website arbitrary OSM data (AKA whatever the editor has changed); at a minimum, I'd have to spin up an instance on-device.

I could use it as a "status as of whenever OsmAnd updated their backend routing db".

Thanks for the information.

[eneerhut]

Is there anything I can do to help with the plugin Taylor? Let me know if mini-PRD defining the features of the routing plugin would be helpful.

Replying to taylor.smock:

Problem:
Mappers attempting to debug why a routing engine does stupid things or takes poor routes.

Solution:
Show mappers a route with route instructions as they edit live.

Goals:

1. Use production routing engines
   • ​valhalla
   • ​OsmAnd
   • ​OSRM
   • ​Organic Maps
2. Show routing instructions in JOSM for given modes (pedestrian, biking, vehicle, and possibly public transport)
3. [Stretch goal] If routing engine supports it, show weights for ways in given mode; this should allow debugging why a different route was not chosen.

[taylor.smock]

I don't think so; I've got the UI up and running, and I'm now working on building valhalla for different platforms.

See ​https://github.com/tsmock/routing2 for the test repo. It will be deleted and cleaned up prior to when I notify some community channels about its availability. There are way too many commits to 'change foo to see if that works in GH actions'.

Right now, the plugin is doing the following:

• Calling the valhalla CLI with a hard-coded set of parameters (example, uses car instead of pedestrian)
• The start/end location can be specified in UI (as lat/lon)
• The route instructions are shown with additional information
• The selected instruction will also be highlighted in the map
• Uses the current data layer for routing (so users can see livish what happens when they change a maxspeed, for example -- livish since it takes 10-30s to rebuild the routing graph and generate the route)

What needs to happen prior to making the plugin available for public testing:

• Compile valhalla for:
  • mac
    • x86
    • arm64
  • Windows
    • x86
  • Linux
    • x86

Right now, I don't think Windows and Linux have significant numbers of arm users. For linux, I will (probably) eventually look into something like ​https://icculus.org/fatelf/ , but x86 support should be "ok" for now.

I do think I'm almost done getting the compilation steps done for mac -- I've got the x86 and arm64 versions built, and am currently working on merging them into a universal "fat" binary.

Once I get those built, I'll then have to decide whether or not to make separate plugins for each platform. Example:

• routing2: Common java files, depends upon virtual dependency routing2-valhalla. Hard dependency may be removed in the future, if other routing engines are supported.
• routing2-valhalla: Virtual dependency fulfilled by platform-specific jar files with the requisite version of valhalla.

For community members reading along, the plugin is not ready yet. Specific problems I'm currently working on:

• You need to have built valhalla and installed it yourself
• Really poor route rendering -- it is "just" a green line with red start/stop points right now

[taylor.smock]

Quick update as related to comment:5:

• Should work on mac/Linux. I'm having issues with Windows.
  • mac/Linux builds are statically linked, which means large (200mb) downloads. Files are extracted during the download, so there shouldn't be too much memory/disk usage from the tar file. I might change this to have a validation step to make certain the tar files weren't tampered with since they were produced.
• I've decided to download binaries from github into the JOSM cache directory. I may stop building fat mac binaries since I don't have to worry about distributing the binaries in a separate plugin that does not have the ability to be filtered on platform arch.
• Still has poor route rendering, but it is better than it was in comment:5. Of specific note, it is now translucent, but that can be modified per-layer with standard layer controls.

Most of my time has been spent troubleshooting Windows. Hard part is I specifically don't want to set up a dev environment on it, since that might mean "it works on my machine" when it doesn't on others.

[gaben]

Do you have a screenshot of the current state? It sounds promising, but at the moment I'd like to avoid making my hands dirty.

[taylor.smock]

I was going to, but now I'm having issues with the pbf plugin. I'll upload some screenshots as soon as I figure out why it can't find crosby.binary.BinarySerializer in the pbf jar.

[gaben]

I've already downloaded it, but okay, waiting for the new version :)

[taylor.smock]

Are you on Windows, macOS, or Linux?

If you are on the latter two, then it should "just work". Assuming the last release had everything build properly, which probably didn't happen since I've been fiddling with the Windows build, trying to figure out why it isn't working (beyond the very generic Failed to parse json request message -- why is it failing to parse the json request?).

Anyway, if you want "auto" updates that you don't have to download manually, add https://github.com/tsmock/routing2/releases/latest/download/updatesite as an additional plugin site (JOSM Preferences -> Plugins -> Configure sites... -- needs Expert Mode). Otherwise, look for the last release with valhalla-*-.tar.gz attachments, which is currently v49. I'll set v49 as the latest since it actually has the required binaries.

[taylor.smock]

Advance warning: the repository state is not stable, and I'll be squashing my debugging experiments before making a wider public release. Don't be surprised if you go from v49 to v15 or something.

[stoecker]

Replying to taylor.smock:

as an additional plugin site (JOSM Preferences -> Plugins -> Configure sites... -- needs Expert Mode).

Wow. You're the first I know who actually uses this feature.

[taylor.smock]

Well, I'm kind of using it for dev purposes. It is much easier to set things up once per machine I'm testing on and have updates taken care of when I go to test. If I didn't have to worry about platform-specific stuff, I wouldn't be using it.

Anyway, once I have the windows build working properly, I'm going to be moving the plugin into the GH JOSM org and adding it to the default plugin list. I haven't added it to the default plugin list yet since I don't want to deal with bug reports from standard users yet, and I know there are issues with the Windows build.

[taylor.smock]

The plugin has been added to wiki:PluginsSource, and the source code has been moved to the JOSM GitHub organization.

IOW, the plugin is ready for a "soft" launch. It still does not do the following:

• Offer multiple methods of transport (car, bike, foot, etc.)

[stoecker]

Add it to StartupPage? Probably a lot of people would like to know about better routing checks.

[taylor.smock]

I'm doing a soft launch -- I want to get feedback from more advanced users before I advertise it widely. I've poked the #routing OSMUS slack channel, with explicit comments that only advanced users should install it for now.

I'm specifically concerned about the distributed binaries -- I tried to ensure that they would "just work" by statically compiling them (hence massive sizes @ ~200mb for Linux and macOS), but there might be some platform specific quirks that I did not see in testing. Like needing to delete some directories and files before each run on Windows. Or needing some additional binary installed because valhalla doesn't embed it, but expects it to be there.

I'm hoping I caught most of the blocking issues, but dev environment != home user environment. And some issues will only be seen on non-dev environments.

I will make a short diary post on OSM about it though, to reflect the current state and what is not currently possible.