Opened 5 months ago
Last modified 4 months ago
#23902 new defect
JOSM Imagery layers outdated / sync with osmlab/editor-layer-index
| Reported by: | rhythmicbalancer | Owned by: | team |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | External imagery source | Version: | |
| Keywords: | Cc: | stoecker |
Description
Hi, I recently noticed that the IndianaMap Orthoimagery layers are broken. It seems the server has gone offline or is otherwise inaccessible. However, this has already been fixed upstream by the osmlab project. Perhaps other problems have been fixed and new sources added since the last time this data was synchronized. Documentation states that ImageryCompare is used to detect and "manually sync" the sources. It would seem this needs to be done again :)
Relative:URL: ^/trunk Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b Last:Changed Date: 2024-08-28 22:00:04 +0200 (Wed, 28 Aug 2024) Revision:19207 Build-Date:2024-08-29 01:30:31 URL:https://josm.openstreetmap.de/svn/trunk Identification: JOSM/1.5 (19207 en) Linux Freedesktop SDK 23.08 (Flatpak runtime) Memory Usage: 2048 MB / 8192 MB (1799 MB allocated, but free) Java version: 21.0.4+2, Flathub, OpenJDK 64-Bit Server VM Look and Feel: javax.swing.plaf.metal.MetalLookAndFeel Screen: :0.0 1600x900x[Multi depth]@60Hz (scaling 1.00×1.00) Maximum Screen Size: 1600×900 Best cursor sizes: 16×16→16×16, 32×32→32×32 Environment variable LANG: en_US.UTF-8 System property file.encoding: UTF-8 System property sun.jnu.encoding: UTF-8 Locale info: en_US Numbers with default locale: 1234567890 -> 1234567890 Desktop environment: GNOME VM arguments: [-Djosm.restart=true, -Djava.net.useSystemProxies=true] Plugins: + Mapillary (1770.1) + RoadSigns (36316) + apache-commons (36282) + ejml (36176) + geotools (36273) + jackson (36273) + jaxb (36118) + jts (36004) + opendata (36327) + utilsplugin2 (36298) Tagging presets: + https://raw.github<user.name>content.com/yopaseopor/traffic_signs_preset_JOSM/master/US.zip Map paint styles: - https://josm.openstreetmap.de/josmfile?page=Styles/Enhanced_Lane_and_Road_Attributes&zip=1 + https://raw.github<user.name>content.com/yopaseopor/traffic_signs_style_JOSM/master/US.zip Last errors/warnings: - 05713.306 W: Can't use layer Thermal Anomalies and Fires (All, Best Available, MODIS, Aqua+Terra) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.307 W: Can't use layer Thermal Anomalies and Fires (All, Best Available, MODIS, Terra) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.307 W: Can't use layer Thermal Anomalies and Fires (Day, Best Available, MODIS, Aqua) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.307 W: Can't use layer Thermal Anomalies and Fires (Day, Best Available, MODIS, Aqua+Terra) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.308 W: Can't use layer Thermal Anomalies and Fires (Day, Best Available, MODIS, Terra) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.308 W: Can't use layer Thermal Anomalies and Fires (Night, Best Available, MODIS, Aqua) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.308 W: Can't use layer Thermal Anomalies and Fires (Night, Best Available, MODIS, Aqua+Terra) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile - 05713.309 W: Can't use layer Thermal Anomalies and Fires (Night, Best Available, MODIS, Terra) because no supported formats were found. Layer is available in formats: application/vnd.mapbox-vector-tile
Attachments (0)
Change History (9)
follow-up: 7 comment:1 by , 5 months ago
| Cc: | added |
|---|
comment:2 by , 5 months ago
| Component: | Wiki content → External imagery source |
|---|
comment:3 by , 4 months ago
Seems loading iD data causes an exception:
Do ant imageryindexdownload in josm/core:
BUILD FAILED
/home/stoecker/sources/josm/core/build.xml:883: The following error occurred while executing this line:
/home/stoecker/sources/josm/core/build.xml:838: jakarta.json.stream.JsonParsingException: JsonParser#getArray() or JsonParser#getArrayStream() is valid only for START_ARRAY parser state. But current parser state is START_OBJECT
at org.eclipse.parsson.JsonReaderImpl.readArray(JsonReaderImpl.java:104)
at SyncEditorLayerIndex.loadELIUsers(SyncEditorLayerIndex.java:401)
at SyncEditorLayerIndex.main(SyncEditorLayerIndex.java:144)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:220)
at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:155)
at org.apache.tools.ant.taskdefs.Java.run(Java.java:892)
at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:232)
at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:136)
at org.apache.tools.ant.taskdefs.Java.execute(Java.java:109)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
at org.apache.tools.ant.Task.perform(Task.java:350)
at org.apache.tools.ant.Target.execute(Target.java:449)
at org.apache.tools.ant.Target.performTasks(Target.java:470)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1401)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36)
at org.apache.tools.ant.Project.executeTargets(Project.java:1264)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437)
at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:106)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
at org.apache.tools.ant.Task.perform(Task.java:350)
at org.apache.tools.ant.Target.execute(Target.java:449)
at org.apache.tools.ant.Target.performTasks(Target.java:470)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1401)
at org.apache.tools.ant.Project.executeTarget(Project.java:1374)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1264)
at org.apache.tools.ant.Main.runBuild(Main.java:818)
at org.apache.tools.ant.Main.startAnt(Main.java:223)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)
Caused by: java.lang.IllegalStateException: JsonParser#getArray() or JsonParser#getArrayStream() is valid only for START_ARRAY parser state. But current parser state is START_OBJECT
at org.eclipse.parsson.JsonParserImpl.getArray(JsonParserImpl.java:135)
at org.eclipse.parsson.JsonReaderImpl.readArray(JsonReaderImpl.java:102)
... 37 more
comment:4 by , 4 months ago
P.S. Note that page ImageryCompare has a link to mapseli, which contains the ELi data as Copy&Paste XML for this wiki ;-)
Will work again after the crash is fixed...
comment:6 by , 4 months ago
Thanks. I was going to look into it after I finished running tests for a change I'm making for #23908.
comment:7 by , 4 months ago
Thank you for the fast responses!
Replying to taylor.smock:
It is a wiki. You can edit it.
Of course, and I'll be happy to help, I simply didn't want to duplicate a lot of effort if there's a script that would help :)
What about naming (in)consistency? Preferably they would all be named similarly because they come from the same source (only differ by year), but I noticed ELI has a deviation there, and I wonder if I should adhere to the old JOSM layer names too. Perhaps only the unique IDs are important?
comment:8 by , 4 months ago
Yes, the unique ids are the most important bit. Everything else can (realistically) change. With that said, keeping the same name as ELI would help with analysis of imagery used in changesets in the future.
The script we use to look for differences will have warnings about different names, different urls, etc.
comment:9 by , 4 months ago
Working on this. Currently looking into the zoom levels, perhaps I can find that from the source tile metadata.
It is a wiki. You can edit it.
We do have a job that checks for stuff like that and updates a wiki page with the output (see wiki:ImageryCompare). It looks like it is broken -- @stoecker: do we have a stack trace I can look at?
In any case, the problem is "manually sync" -- we have to take time from other work to do that. If you have the interest, I would recommend doing it yourself. The wiki page that needs to be edited is at Maps/USA States#IndianaMapOrthoimagery2016.
Anyway, here are the steps I would take to update the URLs:
historicphotoforcategory. Here is a diff (yes, I did the work but didn't apply it, but I'm trying to get someone (specifically you) with interest in the specific endpoints involved).Version
photo</category>maps.indiana.edu/arcgis/services/Imagery/Orthos_2016/MapServer/WmsServer?FORMAT=image/jpeg&TRANSPARENT=FALSE&VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&LAYERS=0&STYLES=&SRS={proj}&WIDTH={width}&HEIGHT={height}&BBOX={bbox}]]></url>photo</category>maps.indiana.edu/arcgis/services/Imagery/Orthos_2017/MapServer/WmsServer?FORMAT=image/jpeg&TRANSPARENT=FALSE&VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&LAYERS=0&STYLES=&SRS={proj}&WIDTH={width}&HEIGHT={height}&BBOX={bbox}]]></url>photo</category>maps.indiana.edu/arcgis/services/Imagery/Orthos_2018/MapServer/WmsServer?FORMAT=image/jpeg&TRANSPARENT=FALSE&VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&LAYERS=0&STYLES=&SRS={proj}&WIDTH={width}&HEIGHT={height}&BBOX={bbox}]]></url>imagery-xml-boundspluginIndiana_Current_Imageryinto JOSM. You will want to copy the generated URL in field 6 of the dialog.Selectionpane and left-click onXML Imagery Bounds<entry>to</entry>) and paste it into wiki:"Maps/USA States" below the 2018 imagery from Indiana. At this point, you will need to then modify the<url></url>section to have the appropriate URL information (which you should have from step 2)<category>to the entry ofphoto(<category>photo</category>).You'll note that this has a different root URL (to a "cache" server) which presumably will have better overall performance than the fallback URL.
I would double-check the max zoom though -- the cache server indicates that there are 24 LoD, so the max zoom should be 23 or 24 instead of 19.
Again, the problem is the "manually sync" bit. The time I took responding is more time than it would take to fix the issue, but I'm hoping that it will increase the number of people who can and will update the wiki if there is a perceived need.