#18053 closed defect (fixed)
No JSON response, but XML for Wikidata Action API error messages: can't be decoded
| Reported by: | gaben | Owned by: | floscher |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | Plugin wikipedia | Version: | latest |
| Keywords: | Cc: | simon04, gaben |
Description
What steps will reproduce the problem?
- Use Wikipedia plugin normally (add wiki tags with the button).
- Add tags until it crashes, because it doesn't happen every time (could be wiki server timeout or so?)
What is the expected result?
No crash, JOSM allow me to use other plugins as well.
What happens instead?
Wikipedia plugin crashes, then JOSM wants to disable nearly all the remaining plugins.
URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2019-08-11 22:00:20 +0200 (Sun, 11 Aug 2019)
Build-Date:2019-08-12 01:30:56
Revision:15296
Relative:URL: ^/trunk
Identification: JOSM/1.5 (15296 hu) Windows 10 64-Bit
OS Build number: Windows 10 Pro 1809 (17763)
Memory Usage: 844 MB / 1820 MB (276 MB allocated, but free)
Java version: 1.8.0_221-b11, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 1920x1200
Maximum Screen Size: 1920x1200
VM arguments: [-Djava.security.manager, -Djava.security.policy=file:<java.home>\lib\security\javaws.policy, -DtrustProxy=true, -Djnlpx.home=<java.home>\bin, -Djnlpx.origFilenameArg=%UserProfile%\AppData\LocalLow\Sun\Java\Deployment\cache\6.0\31\583aa85f-25d43383, -Djnlpx.remove=false, -Djava.util.Arrays.useLegacyMergeSort=true, -Djnlpx.heapsize=NULL,2048m, -Djnlpx.splashport=58991, -Djnlp.application.href=https://josm.openstreetmap.de/download/josm-latest.jnlp, -Djnlpx.jvm=<java.home>\bin\javaw.exe]
Dataset consistency test: No problems found
Plugins:
+ FixAddresses (34977)
+ OpeningHoursEditor (34977)
+ apache-commons (34908)
+ buildings_tools (34982)
+ continuosDownload (82)
+ ejml (35049)
+ geojson (124)
+ geotools (34908)
+ jaxb (35014)
+ jts (35064)
+ opendata (34997)
+ reverter (35084)
+ tag2link (35070)
+ tageditor (34977)
+ turnlanes-tagging (281)
+ utilsplugin2 (34977)
+ wikipedia (v1.1.1)
Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/Maxspeed&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransport&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Surface-DataEntry&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Surface&zip=1
Validator rules:
+ https://josm.openstreetmap.de/josmfile?page=Rules/OsmoseValidations&zip=1
Last errors/warnings:
- W: No configuration settings found. Using hardcoded default values for all pools.
- W: The JSON response from the Wikidata Action API can't be decoded!: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
- E: Handled by bug report queue: java.lang.RuntimeException: java.io.IOException: The JSON response from the Wikidata Action API can't be decoded!. Indok: java.io.IOException: The JSON response from the Wikidata Action API can't be decoded!. Indok: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
- W: Figyelmeztetés - <html>A JOSM nem talált információt az alábbi bővítményekről:<ul><li>apache-commons</li><li>geotools</li><li>tageditor</li><li>opendata</li><li>jts</li><li>tag2link</li><li>ejml</li><li>FixAddresses</li><li>reverter</li><li>continuosDownload</li><li>geojson</li><li>jaxb</li><li>turnlanes-tagging</li><li>utilsplugin2</li><li>OpeningHoursEditor</li><li>buildings_tools</li><li>wikipedia</li></ul>A bővítmények nem lesznek betöltve.</html>
=== REPORTED CRASH DATA ===
BugReportExceptionHandler#handleException:
No data collected.
Warning issued by: BugReportExceptionHandler#handleException
=== STACK TRACE ===
Thread: AWT-EventQueue-2 (46) of javawsApplicationThreadGroup
java.lang.RuntimeException: java.io.IOException: The JSON response from the Wikidata Action API can't be decoded!
at org.wikipedia.WikipediaApp.getWikidataForArticles0(WikipediaApp.java:304)
at org.wikipedia.WikipediaApp.resolveWikidataItems(WikipediaApp.java:256)
at org.wikipedia.WikipediaApp.lambda$getWikidataForArticles$10(WikipediaApp.java:244)
at org.wikipedia.tools.ListUtil.processInBatches(ListUtil.java:29)
at org.wikipedia.tools.ListUtil.processInBatches(ListUtil.java:34)
at org.wikipedia.WikipediaApp.getWikidataForArticles(WikipediaApp.java:243)
at org.wikipedia.actions.FetchWikidataAction$PrimitivesWithWikipedia.updateWikidataIds(FetchWikidataAction.java:131)
at org.wikipedia.actions.FetchWikidataAction$Fetcher.realRun(FetchWikidataAction.java:75)
at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:94)
at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:142)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: The JSON response from the Wikidata Action API can't be decoded!
at org.wikipedia.api.ApiQueryClient.wrapReadDecodeJsonExceptions(ApiQueryClient.java:119)
at org.wikipedia.api.ApiQueryClient.query(ApiQueryClient.java:66)
at org.wikipedia.WikipediaApp.getWikidataForArticles0(WikipediaApp.java:298)
... 12 more
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: (GZIPInputStream); line: 1, column: 2]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:669)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:567)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2624)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:826)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:723)
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4141)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4000)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3070)
at org.wikipedia.api.ApiQuery.lambda$new$0(ApiQuery.java:18)
at org.wikipedia.api.ApiQuery.deserialize(ApiQuery.java:56)
at org.wikipedia.api.ApiQueryClient.query(ApiQueryClient.java:64)
... 13 more
Attachments (1)
Change History (35)
comment:1 by , 5 years ago
follow-up: 5 comment:4 by , 5 years ago
| Summary: | JSON response from the Wikidata Action API can't be decoded → No JSON response, but XML for Wikidata Action API error messages: can't be decoded |
|---|
#18046 has a slightly different message and proper stack trace for the same issue: Error messages are in XML and not JSON.
comment:5 by , 5 years ago
Replying to stoecker:
#18046 has a slightly different message and proper stack trace for the same issue: Error messages are in XML and not JSON.
Also see my comment here: ticket:17832#comment:17
Maybe the Wikidata API changed recently?
comment:6 by , 5 years ago
| Cc: | added |
|---|
Simon, do you know/remember why the WikidataActionApiQuery uses POST? I find it strange. Right now it doesn't work (but works with a simple GET). Did the MediaWiki API used to allow POST before?
comment:7 by , 5 years ago
POST requests seem to have been introduced in https://gitlab.com/JOSM/wikipedia/commit/4f321e0474987b27b475eacf0ca90c1834b91924. Not exactly sure, why.
comment:8 by , 5 years ago
| Priority: | normal → major |
|---|
comment:10 by , 5 years ago
IIRC, I switched to the POST method, because I had potentially very long arguments for some queries (probably multiple wikipedia page titles that I wanted to validate or something like that). So to make sure I don't hit a URL length limit, I then chose to use POST.
But I'll look into this over the weekend and hopefully find a way to fix this.
comment:11 by , 5 years ago
The API works with POST requests if the arguments are supplied using JSON: https://www.mediawiki.org/wiki/API:Tutorial#How_to_use_it
Could be some or all arguments are given as query string, but I don't remember exactly, have to look it up.
comment:12 by , 5 years ago
Maybe this can be useful: https://phabricator.wikimedia.org/T212988#4859184
follow-up: 17 comment:13 by , 5 years ago
This hopefully fixes the issue: https://github.com/JOSM/wikipedia/commit/883dbbad46764e4a97e98149235e8f335eec2761
Vincent, can you craft a new plugin release? (The many Gradle tasks provided intimidate me…)
Eventually we could adhere to https://www.mediawiki.org/wiki/API:Tutorial w.r.t.
For a POST request, format the parameters in JSON format.
comment:14 by , 5 years ago
Sure, I'll do it tomorrow evening if Florian doesn't beat me to it (not used to Gradle neither)
comment:16 by , 5 years ago
| Reporter: | changed from to |
|---|
comment:17 by , 5 years ago
Replying to simon04:
This hopefully fixes the issue: https://github.com/JOSM/wikipedia/commit/883dbbad46764e4a97e98149235e8f335eec2761
It seems to work, thanks.
comment:18 by , 5 years ago
I'm at it, improving some other things as well, will release the new version later today. Then it should be a matter of just pushing a git-tag to the GitLab repository (I sent invitations to Don-vip and simon04).
comment:20 by , 5 years ago
I did hit an issue with the newest gradle-josm-plugin, so the release will need just a little more time.
follow-up: 24 comment:21 by , 5 years ago
Sorry for the delay, but now the version v1.1.2 of the wikipedia plugin is released. See the release notes.
For anyone wanting to make a release in the future, just push a (annotated, or better even a GPG-signed) git-tag to the GitLab repo, the release will be created fully automatically by the CI.
comment:22 by , 5 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Don't be sorry, thanks a lot guys for the fix and the release!
follow-up: 25 comment:24 by , 5 years ago
Replying to floscher:
For anyone wanting to make a release in the future, just push a (annotated, or better even a GPG-signed) git-tag to the GitLab repo, the release will be created fully automatically by the CI.
I cannot push to GitLab
$ git push gitlab master ... remote: GitLab: You are not allowed to push code to protected branches on this project. To gitlab.com:JOSM/plugin/wikipedia.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'git@gitlab.com:JOSM/plugin/wikipedia.git'
comment:25 by , 5 years ago
Replying to simon04:
Replying to floscher:
For anyone wanting to make a release in the future, just push a (annotated, or better even a GPG-signed) git-tag to the GitLab repo, the release will be created fully automatically by the CI.
I cannot push to GitLab
$ git push gitlab master ... remote: GitLab: You are not allowed to push code to protected branches on this project. To gitlab.com:JOSM/plugin/wikipedia.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'git@gitlab.com:JOSM/plugin/wikipedia.git'
I thought you could, but didn't consider that the master branch and tags were protected so only maintainers can push them. Now I upgraded you from developer to maintainer in the Gitlab project.
follow-up: 27 comment:26 by , 5 years ago
Jenkins job fails with org.wikidata.wdtk#wdtk-testing;0.11.1: not found, see https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/jdk=JDK8/1852/consoleText
comment:27 by , 5 years ago
Replying to GerdP:
Jenkins job fails with
org.wikidata.wdtk#wdtk-testing;0.11.1: not found, see https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/jdk=JDK8/1852/consoleText
This is caused, because it is a transitive dependency of org.wikidata.wdtk:wdtk-wikibaseapi:0.11.1, but the artifact for version 0.11.1 of org.wikidata.wdtk:wdtk-testing: is missing from Maven central.
I opened an issue on GitHub for this: https://github.com/Wikidata/Wikidata-Toolkit/issues/495
Gradle doesn't even pull in this dependency (I guess because the pom.xml file defines it to be in test scope). So I tried to work around this by defining the Ivy dependencies with conf="compile": 2ddde2a Let Ivy exclude test dependencies
comment:28 by , 5 years ago
Seems not to work. I get this for r580:
BUILD FAILED C:\josm\plugins\wikipedia\build.xml:43: syntax errors in ivy file: java.text.ParseException: Problem occurred while parsing ivy file: Cannot add dependency 'com.fasterxml.jackson.core#jackson-databind;2.11.0' to configuration 'compile' of module org.openstreetmap.josm.plugins#wikipedia;working@Gerd-PC because this configuration doesn't exist! in file:/C:/josm/plugins/wikipedia/ivy.xml
comment:29 by , 5 years ago
I think you look at \josm\core\ivy.xml instead of \josm\plugins\00_core_tools\ivy.xml
comment:30 by , 5 years ago
To be honest, I looked at neither, I modified the ivy.xml file in the wikipedia plugin: https://gitlab.com/JOSM/plugin/wikipedia/-/blob/master/ivy.xml
I don't really have experience with Ivy configurations. Having now looked a bit more into it, something like conf="default->default" or conf="compile->default" should work better, right?
comment:31 by , 5 years ago
Don't know. I also have no experience with ivy. I also don't understand what you try to do. I hope simon04 knows more.
comment:32 by , 5 years ago
I try to do the following:
This is one of the dependencies in the wikipedia plugins ivy.xml:
<dependency org="org.wikidata.wdtk" name="wdtk-wikibaseapi" rev="0.11.1" />
It has a transitive dependency on org.wikidata.wdtk:wdtk-wikibaseapi:0.11.1, which does not exist.
This is an excerpt from its pom.xml:
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>wdtk-datamodel</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>wdtk-util</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${apacheHttpVersion}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>wdtk-testing</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
I want to tell Ivy to exclude the last of these dependencies by telling it that it should only get the "normal" dependencies and not ones with scope test.
comment:33 by , 5 years ago
This seems to help as ant dist works
-
ivy.xml
1 1 <ivy-module version="2.0"> 2 2 <info organisation="org.openstreetmap.josm.plugins" module="wikipedia"/> 3 3 <dependencies> 4 <dependency org="com.fasterxml.jackson.core" name="jackson-databind" rev="2.11.0" conf="compile"/>5 <dependency org="org.wikidata.wdtk" name="wdtk-wikibaseapi" rev="0.11.1" conf="compile" />6 <dependency org="org.wikidata.wdtk" name="wdtk-dumpfiles" rev="0.11.1" conf="compile" />4 <dependency org="com.fasterxml.jackson.core" name="jackson-databind" rev="2.11.0" /> 5 <dependency org="org.wikidata.wdtk" name="wdtk-wikibaseapi" rev="0.11.1" transitive="false" /> 6 <dependency org="org.wikidata.wdtk" name="wdtk-dumpfiles" rev="0.11.1" transitive="false" /> 7 7 </dependencies> 8 8 </ivy-module>
BUT ant test fails. Will attach a log.
And also only one of the wiki tags gets on the OSM object. Probably the
wikidataif I remember correctly.The issue tracker has similar issues, could be duplicate, please check (#17832, #18025).