NPE in org.openstreetmap.gui.jmapviewer.tilesources.TemplatedTMSTileSource.handleTemplate

[AnyFile]

What steps will reproduce the problem?

1. Just started Josm (last stable version 18360, under windows)
2. Hit download button and a windows popped-up with the title "An unexpected exceprion occurred)

What is the expected result?

What happens instead?

I am no longer able to download anything

Please provide any additional information below. Attach a screenshot if possible.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2022-01-02 21:24:43 +0100 (Sun, 02 Jan 2022)
Build-Date:2022-01-02 20:26:19
Revision:18360
Relative:URL: ^/trunk

Identification: JOSM/1.5 (18360 en_GB) Windows 10 64-Bit
OS Build number: Windows 10 Home 2009 (19044)
Memory Usage: 483 MB / 1820 MB (213 MB allocated, but free)
Java version: 1.8.0_321-b07, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1080 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→32×32, 32×32→32×32
System property file.encoding: Cp1252
System property sun.jnu.encoding: Cp1252
Locale info: en_GB
Numbers with default locale: 1234567890 -> 1234567890

Plugins:
+ apache-commons (35893)
+ apache-http (35893)
+ buildings_tools (35908)
+ imagery_offset_db (35893)
+ jna (35893)
+ jts (35893)
+ reverter (35893)
+ tageditor (35893)
+ todo (30306)
+ turnrestrictions (35893)
+ utilsplugin2 (35893)

Last errors/warnings:
- 00005.856 W: Unable to request certificate of https://grca.nat.gov.tw
- 00028.024 E: Handled by bug report queue: java.lang.NullPointerException



=== REPORTED CRASH DATA ===
BugReportExceptionHandler#handleException:
No data collected.

Warning issued by: BugReportExceptionHandler#handleException

=== STACK TRACE ===
Thread: AWT-EventQueue-0 (19) of main
java.lang.NullPointerException
	at java.util.regex.Matcher.appendReplacement(Unknown Source)
	at org.openstreetmap.gui.jmapviewer.tilesources.TemplatedTMSTileSource.lambda$handleTemplate$1(TemplatedTMSTileSource.java:109)
	at org.openstreetmap.gui.jmapviewer.tilesources.TemplatedTMSTileSource.replacePattern(TemplatedTMSTileSource.java:86)
	at org.openstreetmap.gui.jmapviewer.tilesources.TemplatedTMSTileSource.handleTemplate(TemplatedTMSTileSource.java:107)
	at org.openstreetmap.gui.jmapviewer.tilesources.TemplatedTMSTileSource.<init>(TemplatedTMSTileSource.java:78)
	at org.openstreetmap.josm.data.imagery.JosmTemplatedTMSTileSource.<init>(JosmTemplatedTMSTileSource.java:20)
	at org.openstreetmap.josm.gui.layer.TMSLayer.getTileSourceStatic(TMSLayer.java:120)
	at org.openstreetmap.josm.gui.layer.TMSLayer.getTileSourceStatic(TMSLayer.java:101)
	at org.openstreetmap.josm.gui.bbox.JosmMapViewer.imageryInfosToTileSources(JosmMapViewer.java:73)
	at org.openstreetmap.josm.gui.bbox.JosmMapViewer$AbstractImageryInfoBasedTileSourceProvider.getTileSources(JosmMapViewer.java:65)
	at org.openstreetmap.josm.gui.bbox.SlippyMapBBoxChooser.lambda$getAllTileSources$2(SlippyMapBBoxChooser.java:151)
	at java.util.stream.ReferencePipeline$7$1.accept(Unknown Source)
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Unknown Source)
	at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
	at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
	at java.util.stream.ReferencePipeline.collect(Unknown Source)
	at org.openstreetmap.josm.gui.bbox.SlippyMapBBoxChooser.getAllTileSources(SlippyMapBBoxChooser.java:152)
	at org.openstreetmap.josm.gui.bbox.SlippyMapBBoxChooser.<init>(SlippyMapBBoxChooser.java:117)
	at org.openstreetmap.josm.gui.download.SlippyMapChooser.<init>(SlippyMapChooser.java:35)
	at org.openstreetmap.josm.gui.download.DownloadDialog.buildMainPanel(DownloadDialog.java:148)
	at org.openstreetmap.josm.gui.download.DownloadDialog.<init>(DownloadDialog.java:263)
	at org.openstreetmap.josm.gui.download.DownloadDialog.<init>(DownloadDialog.java:251)
	at org.openstreetmap.josm.gui.download.DownloadDialog.getInstance(DownloadDialog.java:92)
	at org.openstreetmap.josm.actions.DownloadAction.actionPerformed(DownloadAction.java:40)
	at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
	at java.awt.Component.processMouseEvent(Unknown Source)
	at javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.awt.Component.processEvent(Unknown Source)
	at java.awt.Container.processEvent(Unknown Source)
	at java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Window.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$500(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

[AnyFile]

If I used the Download in current view I was able to download.

I completeley removed the directory C:\Users\...my_user_name...\AppData\Local\JOSM\cache

and now it works again

[taylor.smock]

Best guess: the apikey.sites got set to an invalid provider, which meant that when we started asking for API keys, we were getting null results.

JMapViewer apparently does not deal with null values well.

Relevant code:

• source:/trunk/src/org/openstreetmap/josm/io/imagery/ApiKeyProvider.java@head:57#L57​
• source:/osm/applications/viewer/jmapviewer/src/org/openstreetmap/gui/jmapviewer/tilesources/TemplatedTMSTileSource.java@head:103#L103​

TemplatedTMSTileSource uses Matcher.appendReplacement, which does not handle null values, which means we need to do one of two things:

• Never return null from ApiKeyProvider#retrieveApiKey

or

• Modify TemplatedTMSTileSource#handleTemplate to handle null values.

I'm personally inclined to the latter, since JOSM may not be the only user of JMapViewer. At the same time, JMapViewer has a different release process from JOSM, so (a) it might not have a quick release and (b) we might forget to update the version.

[stoecker]

See [35913/osm].

[stoecker]

@Don-vip: New release required. Sadly the release process isn't documented!

[taylor.smock]

That was fast. I just finished getting the sources for JMapViewer when you committed that. :)

It looks like we catch the IAE in source:/trunk/src/org/openstreetmap/josm/gui/bbox/JosmMapViewer.java@head:77#L77​. So we shouldn't have to do anything else, besides do a new release of JMapViewer and update the dependency.

For this specific exception, do we want to check and see if apikey.sites is non-default, and offer to reset it? Or just add it to the message? It doesn't seem like something non-corporate users would be changing (and even corporate users probably wouldn't be changing it).

[stoecker]

Don't know. I didn't think much about that apikey stuff yet. I only activated it now, so that it is actively used (and thus causes issues :-)

[taylor.smock]

Ticket #21846 has been marked as a duplicate of this ticket.

[taylor.smock]

Ticket #21848 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #21845 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #21847 has been marked as a duplicate of this ticket.

[taylor.smock]

Ticket #21849 has been marked as a duplicate of this ticket.

[stoecker]

P.S. If you have time: #21850. Shouldn't be that complicated I think.

[taylor.smock]

Replying to stoecker:

Don't know. I didn't think much about that apikey stuff yet. I only activated it now, so that it is actively used (and thus causes issues :-)

Well, that explains the sudden influx of bugs. :)

[taylor.smock]

Replying to stoecker:

P.S. If you have time: #21850. Shouldn't be that complicated I think.

I'll take a look at that -- I've been porting kendzi3d from JOGL to lwjgl, and it will be nice to have something else to look at.

[stoecker]

Replying to taylor.smock:

Replying to stoecker:

Don't know. I didn't think much about that apikey stuff yet. I only activated it now, so that it is actively used (and thus causes issues :-)

Well, that explains the sudden influx of bugs. :)

See #21839 :-)

[skyper]

Nice, I have deleted all imagery entries but OSM-Carto for jmapviewer, years ago, and never faced the bug.

[stoecker]

In 35912/osm:

see #21844 - drop compatibility

[stoecker]

In 35913/osm:

see #21844 - catch missing API key

[Don-vip]

In 35918/osm:

see #21844 - catch RuntimeException

[Don-vip]

In 35920/osm:

see #21844 - release 2.16

[Don-vip]

Sorry, the release process is not described on wiki.
It's unchanged since #8747. It's weird but straitforward:

- upgrade "ReleaseVersion" SVN property at root directory
- svn update
- ant all
- svn commmit

Then launch https://josm.openstreetmap.de/jenkins/job/Nexus-JMapViewer/ manually to publish the jar on our Nexus repository. It takes the release version in parameter:

https://josm.openstreetmap.de/jenkins/job/Nexus-JMapViewer/37/

[Don-vip]

In 18375/josm:

see #21844 - upgrade to JMapViewer 2.16

[stoecker]

Status "blocker" may be a bit to harsh. The fix was not the JMapViewer, but readding the apikey-file on JOSM server. ;-)

[Don-vip]

sorry I did not check what the issue really was, I just saw my mail inbox spammed :D

[stoecker]

In 18379/josm:

see #21844 - increase jmapviewer version in second place

[Don-vip]

Coverity reminds us we need a proper fix:

*** CID 1469640:  Null pointer dereferences  (NULL_RETURNS)
/src/org/openstreetmap/josm/data/imagery/ImageryPatterns.java: 99 in org.openstreetmap.josm.data.imagery.ImageryPatterns.handleApiKeyTemplate(java.lang.String, java.lang.String)()
93         static String handleApiKeyTemplate(final String id, final String url) {
94             if (id != null && url != null) {
95                 final Matcher matcher = PATTERN_API_KEY.matcher(url);
96                 if (matcher.matches()) {
97                     try {
98                         final String apiKey = FeatureAdapter.retrieveApiKey(id);
>>>     CID 1469640:  Null pointer dereferences  (NULL_RETURNS)
>>>     Dereferencing a pointer that might be "null" "apiKey" when calling "replaceAll".
99                         return matcher.replaceAll(apiKey);
100                     } catch (IOException | NullPointerException e) {
101                         // Match rough behavior in JMapViewer TemplatedTMSTileSource, but with better error message.
102                         throw new IllegalArgumentException(tr("Could not retrieve API key for imagery with id={0}. Cannot add layer.", id), e);
103                     }
104                 }
105             }
106             return url;
107         }

[taylor.smock]

Replying to Don-vip:

Coverity reminds us we need a proper fix:

In that case, we'll want to fix the same coverity issue in JMapViewer. I was deliberately trying to keep this code as similar as possible between JMapViewer and ImageryPatterns.

I'll make some changes and commit them in ImageryPatterns, and then we can copy it over to JMapViewer once Coverity gives its stamp of approval.

[taylor.smock]

In 18380/josm:

ImageryPatterns: Fix CID 1469640: Null point dereferences

This uses an Optional.ofNullable, and if the returned API key is null, we throw
an IOException, which has better troubleshooting text.

see #21844