Context Navigation

Modify ↓

#16747 reopened enhancement

Should not download tiles from other zoom levels if not necessary

Reported by:	naoliv	Owned by:	wiktorn
Priority:	normal	Milestone:
Component:	Core imagery	Version:
Keywords:		Cc:	wiktorn

Description

Go to anywhere with a zoom level of 50m in JOSM, enable any TMS layer (I am using "OpenStreetMap Carto" in this example), clear the layer cache and see how JOSM also loads tiles from lower zoom levels:

2018-09-17 11:25:59.863 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236633/160902.png -> 200 (1,56 kB)
2018-09-17 11:25:59.872 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236631/160902.png -> 200 (2,62 kB)
2018-09-17 11:25:59.873 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236631/160903.png -> 200 (103 B)
2018-09-17 11:25:59.877 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236634/160902.png -> 200 (1,10 kB)
2018-09-17 11:25:59.877 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236631/160901.png -> 200 (3,61 kB)
2018-09-17 11:25:59.879 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236633/160904.png -> 200 (103 B)
2018-09-17 11:25:59.879 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236634/160901.png -> 200 (103 B)
2018-09-17 11:25:59.880 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236632/160903.png -> 200 (336 B)
2018-09-17 11:25:59.884 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236634/160900.png -> 200 (103 B)
2018-09-17 11:25:59.888 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236633/160901.png -> 200 (1,26 kB)
2018-09-17 11:25:59.890 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236632/160904.png -> 200 (103 B)
2018-09-17 11:25:59.891 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236635/160902.png -> 200 (3,10 kB)
2018-09-17 11:25:59.891 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236635/160901.png -> 200 (103 B)
2018-09-17 11:25:59.897 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236634/160903.png -> 200 (103 B)
2018-09-17 11:25:59.898 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236634/160904.png -> 200 (103 B)
2018-09-17 11:25:59.898 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236632/160902.png -> 200 (5,93 kB)
2018-09-17 11:25:59.899 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236633/160900.png -> 200 (103 B)
2018-09-17 11:25:59.899 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236632/160900.png -> 200 (2,59 kB)
2018-09-17 11:25:59.900 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236632/160901.png -> 200 (2,27 kB)
2018-09-17 11:25:59.900 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236630/160902.png -> 200 (103 B)
2018-09-17 11:25:59.897 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236633/160903.png -> 200 (352 B)
2018-09-17 11:25:59.902 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/18/236630/160901.png -> 200 (357 B)
2018-09-17 11:26:00.088 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236635/160903.png -> 200 (103 B)
2018-09-17 11:26:00.102 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236635/160900.png -> 200 (103 B)
2018-09-17 11:26:00.103 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/17/118315/80452.png -> 200 (103 B)
2018-09-17 11:26:00.105 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/17/118317/80450.png -> 200 (103 B)
2018-09-17 11:26:00.150 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236630/160903.png -> 200 (103 B)
2018-09-17 11:26:00.151 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/15/29578/20112.png -> 200 (2,59 kB)
2018-09-17 11:26:00.153 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/16/59158/40226.png -> 200 (103 B)
2018-09-17 11:26:00.155 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/16/59158/40225.png -> 200 (4,41 kB)
2018-09-17 11:26:00.156 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236635/160904.png -> 200 (103 B)
2018-09-17 11:26:00.159 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236631/160904.png -> 200 (103 B)
2018-09-17 11:26:00.163 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/17/118316/80452.png -> 200 (103 B)
2018-09-17 11:26:00.163 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/16/59157/40225.png -> 200 (2,50 kB)
2018-09-17 11:26:00.165 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/17/118316/80451.png -> 200 (4,84 kB)
2018-09-17 11:26:00.167 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/17/118316/80450.png -> 200 (3,01 kB)
2018-09-17 11:26:00.167 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/14/14789/10056.png -> 200 (10,5 kB)
2018-09-17 11:26:00.169 INFORMAÇÕES: GET https://c.tile.openstreetmap.org/16/59157/40226.png -> 200 (103 B)
2018-09-17 11:26:00.175 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/17/118315/80450.png -> 200 (3,47 kB)
2018-09-17 11:26:00.177 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/18/236631/160900.png -> 200 (1,70 kB)
2018-09-17 11:26:00.178 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236630/160900.png -> 200 (2,48 kB)
2018-09-17 11:26:00.179 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/17/118315/80451.png -> 200 (1,95 kB)
2018-09-17 11:26:00.303 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/17/118317/80451.png -> 200 (2,86 kB)
2018-09-17 11:26:00.319 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/17/118317/80452.png -> 200 (103 B)
2018-09-17 11:26:00.370 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/15/29578/20113.png -> 200 (103 B)
2018-09-17 11:26:00.380 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/13/7394/5028.png -> 200 (4,91 kB)
2018-09-17 11:26:00.550 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/18/236630/160904.png -> 200 (103 B)
2018-09-17 11:26:00.557 INFORMAÇÕES: GET https://a.tile.openstreetmap.org/15/29579/20112.png -> 200 (23,1 kB)
2018-09-17 11:26:00.565 INFORMAÇÕES: GET https://b.tile.openstreetmap.org/15/29579/20113.png -> 200 (5,89 kB)

Or, in summary, the number of tiles | zoom level that it downloads:

Number | Zoom
  30   |  18
   9   |  17
   4   |  16
   4   |  15
   1   |  14
   1   |  13

From what I understand, JOSM should only load tiles from the current zoom level; also:

1) lower zoom level tiles should only be download if we receive a 404 for the requested tile (or some other kind of error).

2) of course, if a lower level tile is available in the local cache, it could be used/displayed while the requested tiles are being downloaded.

In a test with the Bing layer, in total it dowloads 427.99 kB at this same area (being 287.03 kB for level 18 (67.06%) and 140.96 kB (32.93%) for the other possibly unneeded levels)

I understand that it's faster to display the lower level tiles while the proper level is being downloaded, but when we are using a low speed Internet connection, it only makes things worse (it's also valid when we are using a limited data plan).

Could we at least have an option to enable or not this behavior, please?
ie, it should always implements 1) and 2), but we should be able to choose if we want to download or not the lower level tiles too.

JOSM:

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2018-09-16 10:26:39 +0200 (Sun, 16 Sep 2018)
Revision:14258
Build-Date:2018-09-16 15:50:45
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (14258 pt_BR) Linux Debian GNU/Linux testing (buster)
Memory Usage: 459 MB / 6144 MB (105 MB allocated, but free)
Java version: 10.0.2+13-Debian-1, Oracle Corporation, OpenJDK 64-Bit Server VM
Screen: :0.0 1600x900, :0.1 1280x1024
Maximum Screen Size: 1600x1024
Java package: openjdk-10-jre:amd64-10.0.2+13-1
Java ATK Wrapper package: libatk-wrapper-java:all-0.33.3-21
VM arguments: [-Dawt.useSystemAAFontSettings=gasp]
Dataset consistency test: No problems found

Attachments (2)

imagery_bug.png (740.9 KB ) - added by Don-vip 6 years ago.
imagery_bug2.png (1.6 MB ) - added by Don-vip 6 years ago.

Change History (19)

comment:1 by wiktorn, 6 years ago

Resolution:	→ fixed
Status:	new → closed

In 14260/josm:

Fix unnecessary tile downloads.

Only download tiles from upper levels when we finished downloading all tiles in
currentZoomLevel.

Closes: #16747
See: #14562

comment:2 by wiktorn, 6 years ago

Milestone:	→ 18.09

comment:3 by Don-vip, 6 years ago

I don't know if it's being caused by this change, but now JOSM often "forgets" to download the correct tiles when using remote control (example: http://localhost:8111/load_and_zoom?left=0.2744444&bottom=43.6713472&right=0.2784444&top=43.675347200000004&select=) with imagery layer "BDOrtho IGN" previously loaded:

Build-Date:2018-09-21 20:12:27
Revision:14265
Is-Local-Build:true

Identification: JOSM/1.5 (14265 SVN en) Windows 10 64-Bit
OS Build number: Windows 10 Pro 1803 (17134)
Memory Usage: 1617 MB / 3634 MB (1155 MB allocated, but free)
Java version: 1.8.0_181-b13, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 1920x1080, \Display1 1920x1080, \Display2 1280x1024
Maximum Screen Size: 1920x1080
VM arguments: [-ea, -Dfile.encoding=UTF-8]
Dataset consistency test: No problems found

Plugins:
+ apache-commons (34506)
+ buildings_tools (34572)
+ cadastre-fr (34630)
+ ejml (34389)
+ geotools (34513)
+ jts (34524)
+ utilsplugin2 (34506)

by Don-vip, 6 years ago

Attachment:	imagery_bug.png added

comment:4 by Don-vip, 6 years ago

Also it seems the tile download error management suffers a regression (no retry?):

2018-09-21 21:03:58.807 INFO: RemoteControl received: GET /load_and_zoom?left=0.3150139&bottom=43.7725472&right=0.3190139&top=43.7765472&select= HTTP/1.1
2018-09-21 21:03:58.976 INFO: GET https://api.openstreetmap.org/api/0.6/map?bbox=0.3150139,43.7725472,0.3190139,43.7765472 -> 200
2018-09-21 21:03:59.183 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65651/47776.jpg -> 200 (9.69 kB)
2018-09-21 21:03:59.189 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65650/47776.jpg -> 200 (10.6 kB)
2018-09-21 21:03:59.200 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65652/47776.jpg -> 200 (9.31 kB)
2018-09-21 21:03:59.207 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65651/47777.jpg -> 200 (13.2 kB)
2018-09-21 21:03:59.209 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65649/47776.jpg -> 200 (12.9 kB)
2018-09-21 21:03:59.292 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65648/47775.jpg -> 200 (13.3 kB)
2018-09-21 21:03:59.310 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65653/47776.jpg -> 200 (10.1 kB)
2018-09-21 21:03:59.314 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65652/47777.jpg -> 200 (8.38 kB)
2018-09-21 21:03:59.325 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65648/47776.jpg -> 200 (13.3 kB)
2018-09-21 21:03:59.333 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65648/47774.jpg -> 200 (9.45 kB)
2018-09-21 21:03:59.363 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65650/47777.jpg -> 200 (13.6 kB)
2018-09-21 21:03:59.390 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65654/47777.jpg -> 509
2018-09-21 21:03:59.409 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65648/47772.jpg -> 509
2018-09-21 21:03:59.425 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65653/47777.jpg -> 200 (9.76 kB)
2018-09-21 21:03:59.447 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65648/47777.jpg -> 200 (11.4 kB)
2018-09-21 21:03:59.449 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65648/47773.jpg -> 200 (8.47 kB)
2018-09-21 21:03:59.465 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65654/47776.jpg -> 200 (12.7 kB)
2018-09-21 21:03:59.513 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/17/65649/47777.jpg -> 200 (13.9 kB)
2018-09-21 21:03:59.642 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/16/32824/23886.jpg -> 509
2018-09-21 21:03:59.643 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/15/16413/11944.jpg -> 509
2018-09-21 21:03:59.643 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/16/32827/23888.jpg -> 509
2018-09-21 21:03:59.644 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/14/8206/5972.jpg -> 509
2018-09-21 21:03:59.645 INFO: GET https://proxy-ign.openstreetmap.fr/94GjiyqD/bdortho/12/2051/1493.jpg -> 509
2018-09-21 21:03:59.707 INFO: RemoteControl received: GET /zoom?left=0.3150139&bottom=43.7725472&right=0.3190139&top=43.7765472 HTTP/1.1
2018-09-21 21:03:59.711 INFO: RemoteControl received: GET /import?url=http://osmose.openstreetmap.fr/api/0.2/error/20050157298/fix/0 HTTP/1.1
2018-09-21 21:03:59.731 INFO: GET http://osmose.openstreetmap.fr/api/0.2/error/20050157298/fix/0 -> 200

by Don-vip, 6 years ago

Attachment:	imagery_bug2.png added

comment:5 by wiktorn, 6 years ago

Resolution:	fixed
Status:	closed → reopened

comment:6 by wiktorn, 6 years ago

Owner:	changed from team to wiktorn
Status:	reopened → new

comment:7 by wiktorn, 6 years ago

Resolution:	→ fixed
Status:	new → closed

In 14268/josm:

Retry tile download on transient errors.

Till now there was no strategy to retry tile download. Tiles for which download
finished with error where by design not downloaded again. Because we were
issuing surplus download requests it was not easy to notice, that some tiles
where downloaded at different zoom level. When surplus downloads where removed -
problem became aparent.

Redefine semantics in AbstractTileSourceLayer, that all Tiles that are not
loaded are tried again. Move all logic deciding whether tile has loaded or not
to TMSCachedTileLoaderJob.

Implment in TMSCachedTileLoaderJob following rules:

when HTTP status code is between 400 and 500 - treat errors as permament
when HTTP status code equals or is greater than 500 - then treat error as

temporary

when there was exception in JCSCachedTileLoaderJob, then HTTP status code is

set to 599. Treat this as permament error. This is suboptimal, as we could treat
Socket read timeout and similar as temporary, and UnkownHostException as
permament

The other aproach is to implement retry count in Tile and retry for finite
number of times, preferably with some delay between tries.

Closes: #16747
See: #16743

comment:8 by Don-vip, 6 years ago

Ah, I was convinced we already had a retry mechanism :D Thanks a lot !

comment:9 by wiktorn, 6 years ago

In 14269/josm:

Remove unnecessary colons

See: #16747

comment:10 by wiktorn, 6 years ago

In 14287/josm:

Rollback [14260].

Because of regressions rollback of [14260].

See: #14562, #16747
Closes: #16789

comment:11 by wiktorn, 6 years ago

Milestone:	18.09 → 18.10
Resolution:	fixed
Status:	closed → reopened

comment:12 by Don-vip, 6 years ago

Milestone:	18.10 → 18.11

comment:13 by Don-vip, 6 years ago

Milestone:	18.11 → 18.12

comment:14 by Don-vip, 6 years ago

Milestone:	18.12 → 19.01

comment:15 by Don-vip, 6 years ago

Milestone:	19.01 → 19.02

comment:16 by Don-vip, 6 years ago

Milestone:	19.02 → 19.03

comment:17 by Don-vip, 6 years ago

Milestone:	19.03

Modify Ticket

Change Properties

Summary:
Type:		Priority:
Milestone:		Component:
Version:		Keywords:
Cc:	Set your email in Preferences

Action

leave as reopened The owner will remain wiktorn.

resolve as The resolution will be set. Next status will be 'closed'.

reassign to The owner will be changed from wiktorn to the specified user. Next status will be 'new'.

need info Next status will be 'needinfo'. The owner will be changed from wiktorn to naoliv.

close as duplicate The resolution will be set to duplicate. Next status will be 'closed'. The specified ticket will be cross-referenced with this ticket.

Add Comment

Your email or username:

E-mail address and name can be saved in the Preferences .

You may use WikiFormatting here.

Attachments ↑ Description ↑

Note: See TracTickets for help on using tickets.

Download in other formats:

Context Navigation

#16747 reopened enhancement

Should not download tiles from other zoom levels if not necessary

Description

Attachments (2)

Change History (19)

comment:1 by wiktorn, 6 years ago

comment:2 by wiktorn, 6 years ago

comment:3 by Don-vip, 6 years ago

by Don-vip, 6 years ago

comment:4 by Don-vip, 6 years ago

by Don-vip, 6 years ago

comment:5 by wiktorn, 6 years ago

comment:6 by wiktorn, 6 years ago

comment:7 by wiktorn, 6 years ago

comment:8 by Don-vip, 6 years ago

comment:9 by wiktorn, 6 years ago

comment:10 by wiktorn, 6 years ago

comment:11 by wiktorn, 6 years ago

comment:12 by Don-vip, 6 years ago

comment:13 by Don-vip, 6 years ago

comment:14 by Don-vip, 6 years ago

comment:15 by Don-vip, 6 years ago

comment:16 by Don-vip, 6 years ago

comment:17 by Don-vip, 6 years ago

Modify Ticket

Add Comment

by anonymous

Download in other formats: