#18812 closed defect (fixed)
JOSM sends invalid HTTP Accept headers
| Reported by: | mmd | Owned by: | simon04 |
|---|---|---|---|
| Priority: | normal | Milestone: | 20.03 |
| Component: | Core | Version: | |
| Keywords: | http httpclient headers accept | Cc: |
Description
When downloading OSM raw data via the OSM API 0.6 /map endpoint, JOSM currently sends a non-conforming HTTP Accept header (according to RFC 7231).
See https://github.com/openstreetmap/openstreetmap-website/pull/2485#issuecomment-592441007 for details.
Attachments (0)
Change History (9)
comment:1 by , 5 years ago
comment:3 by , 5 years ago
| Keywords: | http httpclient headers accept added |
|---|
Are you sure the headers are coming from JOSM? With --debug, I get the following console output:
2020-02-29 21:55:57.127 FINE: REQUEST HEADERS: {Accept-Encoding=gzip, deflate}
2020-02-29 21:55:57.307 INFO: GET https://api.openstreetmap.org/api/0.6/map?bbox=11.3826118,47.2644319,11.3867297,47.2661308 -> HTTP/1.1 200 (181 ms)
2020-02-29 21:55:57.308 FINE: RESPONSE HEADERS: {Transfer-Encoding=[chunked], Keep-Alive=[timeout=5, max=100], null=[HTTP/1.1 200 OK], Strict-Transport-Security=[max-age=31536000; includeSubDomains; preload, max-age=31536000; includeSubDomains; preload], Cache-Control=[private, max-age=0, must-revalidate], Server=[Apache/2.4.29 (Ubuntu)], Content-Disposition=[attachment; filename="map.osm"], Connection=[Keep-Alive], Content-Encoding=[gzip], Date=[Sat, 29 Feb 2020 20:55:57 GMT], Content-Type=[text/xml; charset=utf-8], Expect-CT=[max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly", max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly"]}
comment:4 by , 5 years ago
I started josm with -Djavax.net.debug=all and can see those headers in the log. I also tested against a local server as well, so no other user was triggering the issue. Maybe this is some default header added by Java?
comment:5 by , 5 years ago
| Milestone: | → 20.03 |
|---|---|
| Owner: | changed from to |
| Status: | new → assigned |
Setting osm-server.url=http://api.openstreetmap.org/api reveals this HTTP request in Wireshark:
GET /api/0.6/map?bbox=11.3833058,47.2647132,11.3859022,47.2657835 HTTP/1.1 User-Agent: JOSM/1.5 (15950 SVN en_GB) Linux Arch Linux Java/1.8.0_242 Accept-Encoding: gzip, deflate Host: api.openstreetmap.org Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive HTTP/1.1 301 Moved Permanently Date: Sat, 29 Feb 2020 23:21:21 GMT Server: Apache/2.4.29 (Ubuntu) Cache-Control: max-age=31536000 Expires: Tue, 19 Jan 2038 03:14:07 GMT Location: https://www.openstreetmap.org/api/0.6/map?bbox=11.3833058,47.2647132,11.3859022,47.2657835 Content-Length: 385 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="https://www.openstreetmap.org/api/0.6/map?bbox=11.3833058,47.2647132,11.3859022,47.2657835">here</a>.</p> <hr> <address>Apache/2.4.29 (Ubuntu) Server at api.openstreetmap.org Port 80</address> </body></html>
So, yes, it's coming from Java.
Sending Accept=application/xml, */*;q=0.8 should be fine for all https://www.openstreetmap.org/api/0.6 requests?
comment:8 by , 5 years ago
Sending Accept=application/xml, */*;q=0.8 should be fine for all https://www.openstreetmap.org/api/0.6 requests
That should be fine for endpoints returning XML. Did you already test this change with JSON based endpoints, like for OSM Notes? I think this should also work as is, I'm just double checking here.
comment:9 by , 5 years ago
I did; for notes, we're also using the XML format returned by queries such as https://api.openstreetmap.org/api/0.6/notes?limit=1000&closed=7&bbox=11.3373158,47.2402315,11.3993994,47.2657867
Quoting from the Github issue:
What they are currently sending is the following:
There are a number of issues here:
You can try this out for yourself on the Rails console:
Mime::Type.parse("text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2") Traceback (most recent call last): 1: from (irb):8 Mime::Type::InvalidMimeType ("*; q=.2" is not a valid MIME type)