allow to call external validator from JOSM

[jose1711]

For people writing validation rules mapcss may be too limiting for more complex tasks - spatial tests being one example, checks between otherwise unrelated elements being another. On the other hand not everyone is a java programmer and having a choice of using a programming language (python, shell, perl, lua..) of one's choice by only defining input and output format would be really nice.

A rough idea:

• validator gets an input field in which a set of user defined scripts/executables could be defined
• the script is executed and passed the name of .osm file to run tests on, JOSM is blocked while waiting for the validator(s) to return
• script will return a json with a defined structure and the output will appear in validator window
• external validator errors will be detected by treated gracefully (no fatal errors - just fyi)

[taylor.smock]

We would probably want to fix #4626 first (parallelize validator checks), and probably add the ability for the validator to run in the background as the user edits.

With that being said, I'm disinclined to work on this. If you (or someone else) wants to do the work, then we can talk about it. But this seems like something that should be in a plugin. At least initially. This would allow for rapid development of the API for the external validators without having to wait for JOSM-tested releases. Once the API is stable, it could be merged into JOSM.

I still don't like this idea, just because it is going to depend upon users running additional software on their system. But having it available for people to write reference validation implementations would be nice.

[gaben]

Interesting. The core validator Java API is nice, easy to work with. I think it's easier to ask someone for programming a custom test.

With that being said I see two scenarios:

• you'd like to create the rule but currently can't
• you don't think that others are willing to do the work :)

[jose1711]

With that being said I see two scenarios:

• you'd like to create the rule but currently can't
• you don't think that others are willing to do the work :)

that is quite spot on, minor corrections and additions though from my personal point of view:

• i can't create the rule using mapcss due to its limitations
• i am pretty positive no one will pick it up as the rule is useful for a single (and rather small) country
• the motivation to learn java just to achieve a single task is not really that high

because it is going to depend upon users running additional software on their system.

i understand the argument that we don't want to bring any more dependencies. the way i see this is though is that this would be meant for more advanced mappers (who know what they're doing, accept risks of running 3rd party script and what not). and by default no "alien" scripts would be running.

But this seems like something that should be in a plugin.

works for me. frankly, i don't know how this works. is there a "marketplace" where i can raise a request such plugin? or should i just change the component (currently set to Core).

[taylor.smock]

Replying to jose1711:

With that being said I see two scenarios:

• you'd like to create the rule but currently can't
• you don't think that others are willing to do the work :)

that is quite spot on, minor corrections and additions though from my personal point of view:

• i can't create the rule using mapcss due to its limitations
• i am pretty positive no one will pick it up as the rule is useful for a single (and rather small) country
• the motivation to learn java just to achieve a single task is not really that high

Can you describe the rule? It might be more broadly useful than you think.

But this seems like something that should be in a plugin.

works for me. frankly, i don't know how this works. is there a "marketplace" where i can raise a request such plugin? or should i just change the component (currently set to Core).

It pretty much comes down to someone writing the plugin and adding a link to the released jar file on wiki:/PluginsSource . There isn't really a "marketplace" for plugin requests though. You can see if you can find someone interested in an external validator plugin, as people only work on what they are interested in.

[stoecker]

Shouldn't the "scripting" plugin already allow to do what's requested here?

[jose1711]

Replying to taylor.smock:

Replying to jose1711:

With that being said I see two scenarios:

• you'd like to create the rule but currently can't
• you don't think that others are willing to do the work :)

that is quite spot on, minor corrections and additions though from my personal point of view:

• i can't create the rule using mapcss due to its limitations
• i am pretty positive no one will pick it up as the rule is useful for a single (and rather small) country
• the motivation to learn java just to achieve a single task is not really that high

Can you describe the rule? It might be more broadly useful than you think.

I would like to be able to do things like:

• check if all address nodes have addr:street corresponding to a street name (addr:street value in a set of collected name from ways)
• check if address nodes are not too far from a street they belong to
• uniform format of values in addr:housenumber for a specific area (town/village)

But this seems like something that should be in a plugin.

works for me. frankly, i don't know how this works. is there a "marketplace" where i can raise a request such plugin? or should i just change the component (currently set to Core).

It pretty much comes down to someone writing the plugin and adding a link to the released jar file on wiki:/PluginsSource . There isn't really a "marketplace" for plugin requests though. You can see if you can find someone interested in an external validator plugin, as people only work on what they are interested in.

So ideally I should find that someone :-)

[jose1711]

Replying to stoecker:

Shouldn't the "scripting" plugin already allow to do what's requested here?

thank you, looks promising. will check that out!

[taylor.smock]

Replying to jose1711:

I would like to be able to do things like:

• check if all address nodes have addr:street corresponding to a street name (addr:street value in a set of collected name from ways)
• check if address nodes are not too far from a street they belong to
• uniform format of values in addr:housenumber for a specific area (town/village)

Some of those are already in the JOSM MapWithAI plugin (added since the MapWithAI service now supports address datasources). I've been meaning to move them to a specific general purpose "Testing Validator" plugin -- @stoecker made a repo for that for me, which means I need to take the time to move general validations over into that repo.

Specifically:

• Mismatched street/street addresses (MapWithAI) (see wiki:/Help/Preferences/Validator for where to find the preference when the MapWithAI plugin is installed). This does the first two validations you had.
• Address order (MapWithAI) may also be interesting for you, but it needs some work on my end. It checks that the addresses are in logical order, very much not something you want to check in some locations (there are some localities where addresses might actually look like 1 -> 300 -> 5).

For addr:housenumber, I would anticipate most values matching the regex of [0-9/- ]+ (in my local area, I'd use a more specific regex of [0-9]+( [0-9]+/[0-9]+)?), maybe with unicode characters for 1/2, 3/4, etc. And that can be checked with mapcss. You should modify the regex to account for any specialized formatting for the area.