Duplicated node test fails to detect them properly

[jfd553]

I have imported a lot of data and the resulting ways have duplicated noded at intersection
Validator plugin doesn't detect those nodes even if the option is checked in preferences.

I've include a small file exported using JOSM (and checked with the Validator later version)

[jfd553]

osm map sample with described problem

[HamishB <hamish_b@…>]

Hi,

To make a wild guess, I suspect that you are experiencing a floating point precision error.

For example I am working with some roads + bridges datasets which come from the data source as two layers. The roads are delivered with 6 numbers after the decimal place (~11cm for lat), the bridges to 9 (~0.1mm). Perhaps the Validator's "too close node" tool might help? Does that have a threshold setting?

e.g. if( fabs(a - b) < epsilon) ) { do_merge(a,b) }

It might be nice to apply a 1852*60 to the threshold so that the user can specify it in meters (and multiply by cos(lat) for longitude), but as we are working with questions of exact precision here perhaps it is better to leave it all in the realm of lat/lon?

(I'm not really finding that "too close node" tool, but I'm guessing it is right under my nose somewhere.
Validator
Plugin-Mainversion: 3118,
Plugin-Version: 20799
)

cheers,
Hamish

[HamishB <hamish_b@…>]

Hello,

when opening .osm files downloaded then exported using JOSM, Merkaartor, and Xapi I have noticed some differences that, when mixed, cause the Validator to fail to detect duplicate nodes. It's an ascii representation + floating point precision issue.

• JOSM seems to export with %.9g (~1cm)
• Merkaartor seems to save with %.8f (~1mm)
• Xapi seems to export with %.7f (~1cm)

(fwiw JOSM r3208 also fails to open the downloaded Merkaartor 0.14 .osm file due to missing version= tags in nodes)

Hamish

[HamishB <hamish_b@…>]

Hi, a prototype/example python script to merge close nodes within some ground-units (given in meters) threshhold can be found here:

​http://wiki.openstreetmap.org/wiki/Script_for_snapping_nodes_to_nearby_nodes

it's not all that good; mostly what I want to demonstrate with that is maintaining a constant distance threshold over varying latitudes, with threshold given in a meaningful unit.

Hamish

ps- see also #3720, #4959

[e2jk]

Changed title, removed version info: "Validator (18848)"

[simon04]

Most duplicate nodes are listed as "Duplicate nodes in two un-closed ways". This type cannot be fixed automatically by intention (cf. comment never merge nodes from two different non-closed geometries).

What is the (un)expected behaviour?

[HamishB <hamish_b@…>]

Replying to simon04:

This type cannot be fixed automatically by intention
(cf. comment never merge nodes from two different non-closed geometries).

where can I find/read that comment? (or more importantly others like it) in the source code?

In my case for the New Zealand bulk data upload we get many many duplicate nodes from each of the government's 200 feature layers. A closed area of forest next to a closed area of sand for example. Those snap together easy enough.
But also things like a an unclosed fence line or hedgerow way which is along the edge of a closed forest or field area.

The layers I specifically wrote the python script for in comment:3 was for roads and bridge segments (bridges in the original data are defined by ~3 of the road's nodes (in coordinate space, not by node ID), + attributes).

Hamish

[simon04]

Replying to Hamish <hamish_b@…>:

where can I find/read that comment? (or more importantly others like it) in the source code?

See ​http://josm.openstreetmap.de/browser/josm/trunk/src/org/openstreetmap/josm/data/validation/tests/DuplicateNode.java#L403

[NE2]

Replying to simon04:

Most duplicate nodes are listed as "Duplicate nodes in two un-closed ways". This type cannot be fixed automatically by intention (cf. comment never merge nodes from two different non-closed geometries).

What is the (un)expected behaviour?

This is silly. If two roads have dupe nodes, it should be possible to fix them.

[simon04]

Replying to simon04:

Most duplicate nodes are listed as "Duplicate nodes in two un-closed ways". This type cannot be fixed automatically by intention (cf. comment never merge nodes from two different non-closed geometries).

See also commit message of r3784.

@team: I don't get the reason why these errors are excluded from automatic fixes, neither. What is it?

[mrwojo]

Should we close this ticket? Validator now detects many dupe nodes in the attached map.osm, and the Fix button issue is tracked in #5867.

[Don-vip]

Replying to simon04:

Replying to simon04:

Most duplicate nodes are listed as "Duplicate nodes in two un-closed ways". This type cannot be fixed automatically by intention (cf. comment never merge nodes from two different non-closed geometries).

See also commit message of r3784.

@team: I don't get the reason why these errors are excluded from automatic fixes, neither. What is it?

Me neither. I think we could at least merge all untagged nodes.

@Frederik: do you remember why ?

[Don-vip]

In 5995/josm:

fix #4445, fix #5867 - revert r3784: allow merging of duplicate nodes even with unclosed geometries, as it is really needed

[skyper]

Replying to Don-vip:

Replying to simon04:

Replying to simon04:

Most duplicate nodes are listed as "Duplicate nodes in two un-closed ways". This type cannot be fixed automatically by intention (cf. comment never merge nodes from two different non-closed geometries).

See also commit message of r3784.

@team: I don't get the reason why these errors are excluded from automatic fixes, neither. What is it?

Me neither. I think we could at least merge all untagged nodes.

@Frederik: do you remember why ?

Thought I heard about a problem with tiger import and duplicate nodes on different layers without tags (eg a node of a bridge and the node of the river below).

EDT: Sorry, form mentioning this just after your commit but I did not read it before.

[HamishB <hamish_b@…>]

Replying to skyper:

Thought I heard about a problem with tiger import and duplicate
nodes on different layers without tags (eg a node of a bridge
and the node of the river below).

certainly the New Zealand LINZ imports are full of that exact situation.

thanks,
Hamish

[HamishB <hamish_b@…>]

e.g., here's one where the river and bridge got merged:

​http://www.openstreetmap.org/?lat=-44.28185&lon=-176.17744&zoom=17&layers=M