#12562 closed defect (fixed)
USGS Topographic Map Tiles Slow or Fail to Load
| Reported by: | Owned by: | team | |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | Core imagery | Version: | |
| Keywords: | template_report usgs | Cc: |
Description
What steps will reproduce the problem?
- Start JOSM
- Imagery -> USGS Topographic Maps
What is the expected result?
All USGS Topographic Map tiles load quickly and without error
What happens instead?
- Some tiles load very slowly.
- Sometimes an error is displayed "Error loading tile" in place of the tile itself.
- Sometimes a big red "X" is displayed in place of the tile itself.
- Sometimes and error is displayed "No tiles at this zoom level" in place of the tile itself, even though it may later display after panning / zooming to another area and then returning (indicated that there is a tile at that zoom level).
- Sometimes happens with other imagery sources.
- Sometimes a tile that did display previously (and thus should be in the cache) will fail to display when I return to its area after just a minute or so.
Please provide any additional information below. Attach a screenshot if possible.
I have tried:
- Restarting JOSM
- Rebooting PC
- Flushing tile cache, both by right clicking on map and clicking "flush tile cache" as well as manually deleting from hard drive.
- Right click on map, "Show Tile Info" and copying tile url and pasting in browser address bar always quickly shows tile.
- Turning off all other imagery sources seems to help, but doesn't completely solve the problem.
Revision: 8969 Repository Root: http://josm.openstreetmap.de/svn Relative URL: ^/trunk Last Changed Author: Don-vip Last Changed Date: 2015-10-29 22:15:23 +0100 (Thu, 29 Oct 2015) Build-Date: 2015-10-30 02:32:24 URL: http://josm.openstreetmap.de/svn/trunk Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b Last Changed Rev: 8969 Identification: JOSM/1.5 (8969 en) Windows 7 64-Bit Memory Usage: 324 MB / 1776 MB (89 MB allocated, but free) Java version: 1.8.0_73, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM Dataset consistency test: No problems found Plugins: - AddrInterpolation (31772) - InfoMode (31772) - Mapillary (31802) - PicLayer (31772) - apache-commons (31772) - apache-http (31772) - buildings_tools (31772) - ejml (31772) - geotools (31774) - jts (31772) - opendata (31772) - photo_geotagging (31772) - photoadjust (31772) - proj4j (31772) - undelete (31772) - utilsplugin2 (31772) Last errors/warnings: - W: JCS - Silent failure during download: http://c.tile.openstreetmap.us/usgs_scanned_topos/17/27223/49355.png - W: JCS - Silent failure during download: http://c.tile.openstreetmap.us/usgs_scanned_topos/17/27227/49353.png - W: JCS - Silent failure during download: http://a.tile.openstreetmap.us/usgs_scanned_topos/17/27223/49353.png - W: JCS - Silent failure during download: http://b.tile.openstreetmap.us/usgs_scanned_topos/18/54451/98708.png - W: JCS - Silent failure during download: http://b.tile.openstreetmap.us/usgs_scanned_topos/18/54450/98709.png
Attachments (0)
Change History (21)
comment:1 by , 9 years ago
| Component: | Core → Core imagery |
|---|
comment:2 by , 9 years ago
comment:3 by , 9 years ago
| Keywords: | usgs added |
|---|
comment:4 by , 9 years ago
There might be several things going on here. One is that the cache seems to be ignored. Tiles that should be in the cache are not displaying. I did a test. I waited until I had some USGS tiles successfully loaded (it does happen sometimes). I then disconnected from the Internet. I then could pan and zoom around and any tiles that had been displayed, continued to be displayed. When I try the same thing with the Internet connected, frequently tiles that were previously displayed later fail to display. It seems that JOSM has a preference for re-fetching the USGS tiles rather than using the cache.
comment:5 by , 8 years ago
I am having a similar problem. I'm running JOSM 11223 on a Windows 10 machine. I've been mapping extensively in Alaska during the past few weeks and the USGS Topographic Maps were working fine but since yesterday I get HTTP Response code 502 and many tiles do not load or if they do, the resolution is so low that names cannot be read. Flushing the tile cache doesn't help.
AlaskaDave
follow-up: 7 comment:6 by , 8 years ago
If the server returns 502 JOSM tries it's best to show the tiles based on what it had already downloaded on different zoom levels, that's why the names might be unreadable.
What you can try to do is to go to Advanced Preferences and set:
imagery.generic.minimum_expires and imagery.generic.maximum_expires` to extremely large value such as 31536000000 (1 year). This will bump how long a tile will be fetched from disk cache instead of downloading from network, i.e. tiles will be kept fetch from disk for one year.
*warning* this will affect all layers, so for example if you use also OpenStreetMap Mapnik layer that changes very frequently, you will have to remember to flush the cache of this layer manually from time to time.
The problem with USGS service is that it sets short time-to-live for tiles it returns (7 days) when one takes into account the fact, that they actually wont change anytime soon.
follow-up: 8 comment:7 by , 8 years ago
Replying to wiktorn:
What you can try to do is to go to Advanced Preferences and set:
imagery.generic.minimum_expiresandimagery.generic.maximum_expires` to extremely large value such as 31536000000 (1 year). This will bump how long a tile will be fetched from disk cache instead of downloading from network, i.e. tiles will be kept fetch from disk for one year.
What about this solution: If JOSM was not able to download new tiles it keeps the old tiles even if they expired. This way you can keep imagery.generic.maximum_expires at its default low value without to fear to loose tiles due to a temporary bad connection to the tile server.
follow-up: 9 comment:8 by , 8 years ago
Replying to Klumbumbus:
What about this solution: If JOSM was not able to download new tiles it keeps the old tiles even if they expired. This way you can keep
imagery.generic.maximum_expiresat its default low value without to fear to loose tiles due to a temporary bad connection to the tile server.
This logic is already implemented in JOSM and that's why user sees some tiles in different resolution.
If HTTP 502 is a sign of "bandwidth exceeded", then what we need is just to do as few requests towards tile source as we could. If we see HTTP 502 it is already too late.
follow-up: 10 comment:9 by , 8 years ago
Replying to wiktorn:
This logic is already implemented in JOSM and that's why user sees some tiles in different resolution.
If this logic is working properly then the behavior described in comment:4 shouldn't be possible, if I understand it right. JOSM should display the tile from the cache as long as the tile is not successfully completely redownloaded from the server.
If HTTP 502 is a sign of "bandwidth exceeded", then what we need is just to do as few requests towards tile source as we could. If we see HTTP 502 it is already too late.
How do we know the reason for the 502 error is "bandwidth exceeded"? Couldn't it have other reasons?
comment:10 by , 8 years ago
Replying to Klumbumbus:
Replying to wiktorn:
This logic is already implemented in JOSM and that's why user sees some tiles in different resolution.
If this logic is working properly then the behavior described in comment:4 shouldn't be possible, if I understand it right. JOSM should display the tile from the cache as long as the tile is not successfully completely redownloaded from the server.
I guess that this was fixed in [10545]. So my guess that it was possible at that time.
If HTTP 502 is a sign of "bandwidth exceeded", then what we need is just to do as few requests towards tile source as we could. If we see HTTP 502 it is already too late.
How do we know the reason for the 502 error is "bandwidth exceeded"? Couldn't it have other reasons?
That's my wild guess. I was fetching tiles for some time, and when I started getting 502, I was getting it for all further requests. Once I waited for some time, then I could again download some more tiles. That's how I remember that it was looking in February.
comment:12 by , 8 years ago
Just to keep the information why this ticket is open - my feeling is that the best way to solve this is to have possibility to set different minimum expires time per imagery.
follow-up: 14 comment:13 by , 8 years ago
My tiles are loading properly now. I am happy to see the imagery return but am curious as to why it happened in the first place. I forgot to mention that based information in the original ticket I tried to load a tile using its URL: "4. Right click on map, "Show Tile Info" and copying tile url and pasting in browser address bar", but in my case that did not load the tile. That would seem to nullify the bandwidth exceeded argument.
Thanks for the quick feedback.
AlaskaDave
follow-up: 15 comment:14 by , 8 years ago
Replying to daveswarthout@…:
I forgot to mention that based information in the original ticket I tried to load a tile using its URL: "4. Right click on map, "Show Tile Info" and copying tile url and pasting in browser address bar", but in my case that did not load the tile. That would seem to nullify the bandwidth exceeded argument.
That's exactly what makes me think that 502 error is sign of traffic limitation. If you can't get to load tile in JOSM nor in browser then it may mean that you've just hit limit of transfer based on IP address.
follow-up: 16 comment:15 by , 8 years ago
Replying to wiktorn:
Replying to daveswarthout@…:
I forgot to mention that based information in the original ticket I tried to load a tile using its URL: "4. Right click on map, "Show Tile Info" and copying tile url and pasting in browser address bar", but in my case that did not load the tile. That would seem to nullify the bandwidth exceeded argument.
That's exactly what makes me think that 502 error is sign of traffic limitation. If you can't get to load tile in JOSM nor in browser then it may mean that you've just hit limit of transfer based on IP address.
2016-12-19 I'm back to having the same problem. Only low res imagery showing, cannot load the faulting tiles from their URLs (HTTP error 404). The limitation of transfer you mention — who is imposing it? The tiles are hosted by OSM, correct?
Do you know if there is some good, as in easy, way to load a subset of Topos locally so my Alaska work can continue?
follow-up: 17 comment:16 by , 8 years ago
Replying to daveswarthout@…:
2016-12-19 I'm back to having the same problem. Only low res imagery showing, cannot load the faulting tiles from their URLs (HTTP error 404). The limitation of transfer you mention — who is imposing it? The tiles are hosted by OSM, correct?
I'm not so sure, that the tiles are hosted by OSM. The page:
http://wiki.openstreetmap.org/wiki/Foundation/Local_Chapters/United_States/Servers/Imagery
States, that they are *proxied*. Though the WMS to which USGS Topo layer points doesn't serve any layers. You should try to contact administrators of this tile source. Maybe authors of this wiki page or maybe on US forum you can get more help.
But maybe 404 means, that the scan is not available in the area that you're working.
comment:17 by , 8 years ago
Replying to wiktorn:
Replying to daveswarthout@…:
2016-12-19 I'm back to having the same problem. Only low res imagery showing, cannot load the faulting tiles from their URLs (HTTP error 404). The limitation of transfer you mention — who is imposing it? The tiles are hosted by OSM, correct?
I'm not so sure, that the tiles are hosted by OSM. The page:
http://wiki.openstreetmap.org/wiki/Foundation/Local_Chapters/United_States/Servers/Imagery
States, that they are *proxied*. Though the WMS to which USGS Topo layer points doesn't serve any layers. You should try to contact administrators of this tile source. Maybe authors of this wiki page or maybe on US forum you can get more help.
But maybe 404 means, that the scan is not available in the area that you're working.
Usually, error 404 means file not found. I believe the scans are available for the areas I'm working in because I saw good resolution details earlier. I'll check the Wiki pages you mentioned and take it from there. Thanks.
comment:18 by , 8 years ago
USGS topographic maps tiles should load fine again, since we pull them from another source now. More info about this see https://github.com/osmlab/editor-layer-index/issues/151 and https://github.com/osmlab/editor-layer-index/pull/290
I'm not sure if this ticket should be closed or kept open due to comment:12
comment:19 by , 8 years ago
We can close this. Expiration times per imagery I implement together with #13118.
comment:20 by , 8 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
comment:21 by , 8 years ago
Fantastic! The new imagery source is beautiful. Many thanks to all who worked on finding a solution to this thorny problem. Up until yesterday, I had been using copies of the 1:100,000 Alaska maps downloaded from the USGS Topo store to my desktop where I was trying to use them in my mapping. It's very difficult and no substitute for a JOSM layer!
I've checked, it looks like a real problem that needs some better logging on JOSM side. But it looks like the real problem is on the server side. When I do some more map browsing, the server starts returning:
`<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body bgcolor="white">
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.1.19</center>
</body>
</html>
`If you happen to know, what are the limitations of the tile service provider and how we can avoid them, then we can think, how to incorporate them into JOSM.