[Patch] Cannot access HTTPS Dutch WMTS servers without using WebStart or Linux

[Allroads]

Hello,

This wmts url is working in the webstart. Tested by others.
​http://geodata.nationaalgeoregister.nl/tiles/service/wmts/bgtachtergrond?SERVICE=WMTS&request=GetCapabilities

Projection set on Rijksdriehoek RD EPSG:28992 Important!!

Used in webstart it works.
With JOSM .exe .jar, it is not.

Imagery: Also in the available default entries

https://josm.openstreetmap.de/wiki/Maps/Netherlands#PDOKLuchtfotoBeeldmateriaal25cmWMTS

This ortho wmts is not working in Josm .jar and with webstart it is.

It does not matter if service=WMS or set on service=WMTS in the wmts string both does not work.

Josm 11826
Java Version 8 Update 121 1.8.0_121-b13 latest
JOSM is on: No proxy

Started Josm with .bat file
cmd.exe copy past field started error wmts.

2017-04-14 14:20:54.268 INFO: GET https://geodata.nationaalgeoregister.nl/luchtfoto/wmts?&request=GetCapabilities&service=WMS -> !!!
2017-04-14 14:20:54.268 WARNING: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
        at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
        at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
        at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)
        at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
        at sun.security.ssl.Handshaker.processLoop(Unknown Source)
        at sun.security.ssl.Handshaker.process_record(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)
        at org.openstreetmap.josm.tools.HttpClient.connect(HttpClient.java:125)
        at org.openstreetmap.josm.tools.HttpClient.connect(HttpClient.java:75)
        at org.openstreetmap.josm.io.CachedFile.checkLocal(CachedFile.java:472)
        at org.openstreetmap.josm.io.CachedFile.getFile(CachedFile.java:272)
        at org.openstreetmap.josm.io.CachedFile.getInputStream(CachedFile.java:207)
        at org.openstreetmap.josm.data.imagery.WMTSTileSource.getCapabilities(WMTSTileSource.java:331)
        at org.openstreetmap.josm.data.imagery.WMTSTileSource.<init>(WMTSTileSource.java:278)
        at org.openstreetmap.josm.actions.AddImageryLayerAction.convertImagery(AddImageryLayerAction.java:99)
        at org.openstreetmap.josm.actions.AddImageryLayerAction.actionPerformed(AddImageryLayerAction.java:138)
        at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
        at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
        at javax.swing.AbstractButton.doClick(Unknown Source)
        at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
        at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
        at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
        at java.awt.Component.processMouseEvent(Unknown Source)
        at javax.swing.JComponent.processMouseEvent(Unknown Source)
        at java.awt.Component.processEvent(Unknown Source)
        at java.awt.Container.processEvent(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$500(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
        at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
        at sun.security.validator.Validator.validate(Unknown Source)
        at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
        ... 59 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source)
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
        at java.security.cert.CertPathBuilder.build(Unknown Source)
        ... 65 more

[bastiK]

Please add full status report!

[Allroads]

Please provide any additional information below. Attach a screenshot if possible.

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2017-04-02 01:45:00 +0200 (Sun, 02 Apr 2017)
Build-Date:2017-04-02 01:34:50
Revision:11826
Relative:URL: ^/trunk

Identification: JOSM/1.5 (11826 en) Windows 10 64-Bit
Memory Usage: 497 MB / 910 MB (341 MB allocated, but free)
Java version: 1.8.0_121-b13, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 1680x1050, \Display1 1680x1050
Maximum Screen Size: 1680x1050

Plugins:
deleted privat

Tagging presets:
deleted privat

Map paint styles:
deleted privat

Last errors/warnings:
- E: Failed to locate image ''
- W:  Privat: Could not get presets icon 
- E: Failed to locate image ''
- W:  Privat: Could not get presets icon 
- E: Failed to locate image ''
- W:  Privat: Could not get presets icon 
- W: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
- E: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

[bastiK]

Please update imagery list and try again. It is important that the capabilities request URL starts with http:// and not https://.

@team: The root certificate is available on Debian, but apparently not in the Java keystore on Windows:

$ keytool -keystore /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/cacerts -list
[...]
debian:staat_der_nederlanden_root_ca_-_g3.pem, Feb 7, 2016, trustedCertEntry, 
Certificate fingerprint (SHA1): D8:EB:6B:41:51:92:59:E0:F3:E7:85:00:C0:3D:B6:88:97:C9:EE:FC
[...]

[Don-vip]

Replying to bastiK:

@team: The root certificate is available on Debian, but apparently not in the Java keystore on Windows:

Confirmed, see attached file.

[Allroads]

But how is this solved.

Is it done with a new JOSM update?

When?

[Don-vip]

There are 6 ​QuoVadis entries in the Java keystore, but not "Staat der Nederlanden Root". QuoVadis roots have been updated recently, see ​javabug:8145955 but they only include the "standard" root CAs.

Here the CA is "QuoVadis CSP - PKI Overheid CA" which seems to be a system specifi to ​Dutch government.

Should we include it in JOSM?

[bastiK]

We could do that, but the tile URL template that is returned by the capabilities request makes JOSM request the tiles over plain http:

2017-04-14 16:14:13.754 INFO: GET http://geodata.nationaalgeoregister.nl/luchtfoto/wmts?SERVICE=WMTS&REQUEST=GetTile&VERSION=1.0.0&LAYER=2016_ortho25&STYLE=default&FORMAT=image/png8&tileMatrixSet=OGC:1.0:GoogleMapsCompatible&tileMatrix=11&tileRow=677&tileCol=1057 -> 200 (63.0 kB)

So there isn't gained much by making the initial request over https and then fetching the potentially more sensitive data over http.

[bastiK]

Replying to Allroads:

But how is this solved.

By changing the URL in the imagery database from https to http.

Is it done with a new JOSM update?

No, you just need to update the Imagery list in the preferences or wait 24 h for auto-update.

[stoecker]

Replying to Don-vip:

Here the CA is "QuoVadis CSP - PKI Overheid CA" which seems to be a system specifi to ​Dutch government.

Should we include it in JOSM?

I'd say no:

• I don't find it in firefox
• It is a single use case
• The use is solved by the maps update

[bastiK]

If any, we should include the root certificate "staat_der_nederlanden_root_ca_-_g3.pem" (which is included in Firefox and Chrome) and not "QuoVadis CSP - PKI Overheid CA".

[Allroads]

Replying to stoecker:

Replying to Don-vip:

I'd say no:

• I don't find it in firefox

• It is a single use case

single use, I ask on behalve of the whole Dutch community. Not for me, only person.

• The use is solved by the maps update

Not only for the default entries, like ortho photo.

But there many other layers to use licence PD or CC0 1.0
All together in a big list
​https://geodata.nationaalgeoregister.nl/tiles/service/wmts?SERVICE=WMTS&request=GetCapabilities

When Government uses https links for use, metadata, I find that it must be easy as possible to add new wmts entries for everyone. They give in the above link, that should be working directly with https url.
Not everyone knows the http s story, so they are annoyed, thinking the layers do not work.
This gives questions on forum and to the Government. Do we want that?

It happend also to me. Why does the layer do not work.

This is also with other wms layers of the Goverment.

What good certificates are, I do not know, it is not my knowledge.

[stoecker]

Replying to Allroads:

• It is a single use case

single use, I ask on behalve of the whole Dutch community. Not for me, only person.

There is a single user of these certificates, the Dutch government.

Not everyone knows the http s story, so they are annoyed, thinking the layers do not work.

Which would be correct. They do not work reliable with https, as the certificates aren't included in Java (and probably other tools as well).

This gives questions on forum and to the Government. Do we want that?

It was their decision to choose these certificates.

What good certificates are, I do not know, it is not my knowledge.

That's simple - they must be trustworthy enough to be accepted in as many products as possible and they must be included in these products.

We added own certificate storage in JOSM for the cases were Java is behind all the other browsers. But the rule is we do not add anything, which the big browsers don't add and there must be a high enough demand to include a certificate. If there are other possibilities not requiring to include a certificate these should be used.

For now JOSM included the CA's providing free certificates, as these enable normal people to offer https services, so it is in our interest to support these (now only Let's Encrypt) even if Java does not (yet). To support a single government CA I see no real benefit for JOSM.

[Don-vip]

Java does support Let's Encrypt now, since ​8u101 released in July 2016.

Looking at usage statistics, 82.5% of our users use a compatible version:

J        649 ( 5.7%) Java/1.8.0_101
J        120 ( 1.1%) Java/1.8.0_102
J       1124 ( 9.9%) Java/1.8.0_111
J        103 ( 0.9%) Java/1.8.0_112
J       7366 (64.9%) Java/1.8.0_121

And 17% do not:

J          9 ( 0.1%) Java/1.8.0
J         17 ( 0.1%) Java/1.8.0_05
J          9 ( 0.1%) Java/1.8.0_11
J         22 ( 0.2%) Java/1.8.0_20
J        122 ( 1.1%) Java/1.8.0_25
J        144 ( 1.3%) Java/1.8.0_31
J         56 ( 0.5%) Java/1.8.0_40
J        126 ( 1.1%) Java/1.8.0_45
J         81 ( 0.7%) Java/1.8.0_51
J        129 ( 1.1%) Java/1.8.0_60
J         96 ( 0.8%) Java/1.8.0_65
J        246 ( 2.2%) Java/1.8.0_66
J         52 ( 0.5%) Java/1.8.0_71
J          6 ( 0.1%) Java/1.8.0_72
J         97 ( 0.9%) Java/1.8.0_73
J         41 ( 0.4%) Java/1.8.0_74
J        141 ( 1.2%) Java/1.8.0_77
J        461 ( 4.1%) Java/1.8.0_91
J         62 ( 0.5%) Java/1.8.0_92

We will soon be able to remove it as well, see #14652.

[Don-vip]

Back to the point.
The certificate is trusted by (at least) Mozilla, Microsoft and Debian.
Plus, Allroads raises a very good point: the capabilities returned by the HTTP link returns a lot of services in HTTPS.
This seems a nice enhancement to Dutch community to be able to add https maps out of the box, without the need to edit the JOSM wiki.

It's not so different from the inclusion of NTv2 grids which only concerned a few countries.

[Allroads]

What happend, I open a Openstreetmap forum topic, because a men of the Dutch Government said, that the webstart JOSM did show the layers with https. Therefor the question here.
​https://forum.openstreetmap.org/viewtopic.php?id=58039 Dutch.
Now several of us did install the certificate in the JAVA keystore. I exported them from Firefox.
Because we wanted to know, if it works.
https is working and we get the tiles. All good.
You can read how in the forumlink.
StaatderNederlandenRootCA-G2.crt
but also
StaatderNederlandenOrganisatieCA-G2.crt
QuoVadisCSP-PKIOverheidCA-G2.crt

Not everyone, who tag in The Netherlands, who uses JOSM read the Dutch forum. Topic is going down in the list.
The manual solution is not for everyone.

It is not only the layers, who are written in the wiki, the default ones.
Many of us, use/to test, single layers as well and manually give in the wms/wmts.
Copied, nowadays, from metadata the htpps link.
Knowing the s problem, I forget it also, not getting a layer list.
Ouch, thinking I knew it.

[stoecker]

Replying to Don-vip:

Back to the point.
The certificate is trusted by (at least) Mozilla, Microsoft and Debian.

Which one? "QuoVadis CSP - PKI Overheid CA" is not in my list. I found one "Staat der Nederlanden Root CA G2" thought.

[Don-vip]

I was speaking of the root CA, which is "Staat der Nederlanden Root CA G2".

This is also strange that Java, when running with WebStart, accesses the local Windows keystore (which contains this CA) while a standard run with java -jar does not. Hard to understand for end-users.

[Allroads]

One user wrote:
After 11826 installed, JOSM did not gave a pop up anymore with failures but the wmts layer bgtstandaard ( topic start url deep zoom in for layer) was not shown, on screen where tiles should be, in red "Error: Problem loading tile"

The version before 11826, had this popup report.

And
Before that the wmts layer worked.

[Allroads]

When I installed certificate
First I installed only this one.
StaatderNederlandenRootCA-G2.crt
Then tested it and I did get the tiles on the screen.

Then installed the other ones. Maybe not needed.

[Don-vip]

Better solution that should satisfy everyone: load the certificate from Windows keystore if not present in Java keystore. Nothing has to be embedded, just the alias and hash in source code, see attached patch (tested OK).

@Dirk: is it OK for you?

[stoecker]

Looks like a useful solution.

[Don-vip]

In 11943/josm:

fix #14649 - load Dutch Government (G2 & G3) certificates from Windows keystore if not found in Java keystore

[Don-vip]

In 11944/josm:

see #14649 - fix @since

[Don-vip]

In 12241/josm:

see #11924, see #14649 - java 9 does not seem to include Dutch certificates yet, load them from /usr/share/ca-certificates/mozilla (see Debian ca-certificates package)

[Don-vip]

In 12623/josm:

see #14649, see #15178 - load Certigna certificate from platform keystore, needed for cadastre-fr plugin, as it is now used by French government

[Don-vip]

Follow-up: #15992

[taylor.smock]

In 19158/josm:

See #14649: Drop certificate amendment for Netherlands

www.nationaalgeoregister.nl is properly setup to have the trust chain and uses
QuoVadis Root CA 2, which is in the default Java trust store.

Furthermore, geodata.nationaalgeoregister.nl is no longer a registered
subdomain. In addition, our map backgrounds make no references to that subdomain.