TMS failed tiles make CPU load go up indefinitely; remove/readd layer doesn't help

[alv]

This started with r8339, didn't exist with r8109). At least one other user has notices this, too.

What steps will reproduce the problem?

1. Add imagery layer, pan around until one tile remains black or the lower zoom level tile is never replaced for that tile.
2. Notice the computer fan pick up some speed, because something is doing nothing all the time on one core (here: 25% load on a 4 core machine).
3. Panning to a screen with only correctly loaded tiles visible does not help.
4. (Unconfirmed, but it seems the proportion of missing tiles increases, as the recently added #10902 threads keep waiting for the first unsuccessfull tile(s)?)

When loading a tile is never finished, something consumes one core's worth of CPU constantly. Only minimizing JOSM or hiding the imagery layer stops it, but it resumes if reshown; even if JOSM is not in the foreground, it keeps doing nothing with great effort. Removing imagery layer and re-adding it does _not_ help this.

Revision: 8339
Repository Root: http://josm.openstreetmap.de/svn
Relative URL: ^/trunk
Last Changed Author: stoecker
Last Changed Date: 2015-05-07 17:30:43 +0200 (Thu, 07 May 2015)
Build-Date: 2015-05-08 01:31:23
URL: http://josm.openstreetmap.de/svn/trunk
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last Changed Rev: 8339

Identification: JOSM/1.5 (8339 fi) Windows 7 64-Bit
Memory Usage: 477 MB / 892 MB (107 MB allocated, but free)
Java version: 1.8.0_31, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Dataset consistency test: No problems found

Plugins:
- DirectUpload (30892)
- PicLayer (31114)
- editgpx (30892)
- geotools (31068)
- imagery_offset_db (31056)
- jts (31002)
- kendzi3d (1.0.184)
- kendzi3d-jogl (37)
- log4j (30892)
- measurement (30892)
- opendata (31116)
- public_transport (31114)
- reverter (31120)
- turnlanes (30892)
- undelete (30892)
- utilsplugin2 (31120)
- waydownloader (30892)
- wikipedia (31114)

Last errors/warnings:
...
- W: JCS TMS Cache - error creating URL for tile 16/37315/18965@Bing Aerial Maps: Attribution is not loaded yet
- W: No url returned for: 16/37315/18965@Bing Aerial Maps, skipping

[wiktorn]

Few questions that will help me diagnose the problem:

1. Can you explain what do you mean by hiding the imagery layer?
2. Do the tiles continue to load, once the problem appears?
3. Did you sent the error report, when the problem occured?

[alv]

1. hiding = left click in layer list + show/hide
2. most of the tiles do load, but it seems that once the problem starts, "one in x" of the newly visible tiles remain black after that
3. I tried to send the report when the problem occured: I had witnessed it at least twice before, but then I had already kept editing for a long time; this time I did send it as soon as I noticed the CPU load.

[wiktorn]

In 8389/josm:

addresses #11437 - properly pass information about errors during load from cache to upper layers

[wiktorn]

Please check, if this addresses your problem. It reduced the cpu load for situation, when there is tile loading error, but in my tests - the load disappeared when layer was removed, so I'm not sure, if this was your case.

[aceman]

Confirming as reported. A tile can stay black or a tile stays in lower zoom (big pixels) while others are loaded in more detailed zoom. JOSM (actuall the java process on linux AND the X server binary) take ~100% of 1 CPU core.
Hiding the imagery layer solves the problem.

[aceman]

This overloading of the X server also causes other heavy apps to appear sluggish (like Firefox). This is NOT swapping to disk. Probably that the X server can't serve the requests quickly as it is overloaded by JOSM's requests.

[alv]

I'll be testing with josm-latest from now on.

What I meant with "removing imagery layer and re-adding" was that if the cpu load is on and I remove the offending layer, the cpu load stops, but when I re-add that same layer, the extra processing starts right away. I couldn't yet confirm my next finding: nothing new is written to the josm status report "Last errors/warnings" list, the one I left in the original report is just the first warning when the layer is added after startup. This has now (before the change by wiktorn) happened at least once with a non-bing imagery, too.

[wiktorn]

In 8397/josm:

addresses #11437 - introduce infinite queue for tile loading and clear the queue when user pans the map or changes the zoom. Fixing hosts limit is last problem, thay may incur additional load

[wiktorn]

In 8403/josm:

Rework the per host limit, so the queue will never reject the submited job. Also - sort tiles during loading in TMS, so center tiles will be loaded first. closes: #11437