I need help on adding bus routes.

[mrgenie]

I have gpx tracks 150-350km for bus routes.

However, when I select my gpx and do "download from OSM along this track"
with the following settings:

download everything within 50 meters
maximum area per request 3 square kilometer (otherwise it's going to be too many downloads and with my high latency here opposite of the planet to Amsterdam servers going to take ages)

I mean I can select download area 0.01 square km - but with thousands of downloads and my high latency it's going to take hours as well.

I get the following error messages - see attachments.

About that the amount of data I'm downloading is too big.

I asked the OSM support for the servers if they can do something on the restrictions preventing me from adding medium range bus routes but they already replied nothing will be done in this regard. Instead I should just join the discussion groups and discuss my frustration that I can't add bus routes using JOSM due to the restrictions - but it won't be resolved by lifting these limitations of 1980 sizes of 1MB aka floppy disk size restrictions.

So how to mitigate this in JOSM so I can add such bus routes using JOSM?

And I tried using to download objects from the overpass - but if I do that - granted I get all the data - but then I keep seeing these yellowish stripes and can't actually work on that data and upload it - to upload I need to download it twice - again from the api.openstreetmap.org otherwise it'll reject the added bus route changes.

so I'm kinda stuck. The alternative is just stop recording medium range bus routes using gpx and simply don't add them to OSM. - which is what I'm now considering - because 4-5 hours to download some data to add a medium range bus route - I can drink coffee sure but drinking 4-5 hours coffee because every 10 minutes or so I can download another small area along the route - is kinda extremely frustrating.

And it's not like I don't have tons of other data I can add to OSM for which I don't need to download larger areas obviously

[mrgenie]

settings

[mrgenie]

Error message

[mmd]

I asked the OSM support for the servers if they can do something on the restrictions preventing me from adding medium range bus routes but they already replied nothing will be done in this regard. Instead I should just join the discussion groups and discuss my frustration that I can't add bus routes using JOSM due to the restrictions

I think you're misrepresenting my comment in ​https://github.com/openstreetmap/operations/issues/888#issuecomment-1567527335

No one rejected to change any parameters, or increase the download limits. This is a decision to be made by the operations team, which I'm not part of. Since you have closed your Github issue without good reason, nothing will happen here anymore, as expected.

Joining discussions was solely referring to the Overpass query part. Maybe you somehow missed this?

[mmd]

And I tried using to download objects from the overpass - but if I do that - granted I get all the data - but then I keep seeing these yellowish stripes and can't actually work on that data and upload it - to upload I need to download it twice - again from the api.openstreetmap.org otherwise it'll reject the added bus route changes.

This issue was really caused by not using [out:xml]; and out meta; to generate Overpass API responses in OSM XML format, see my comment here: https://josm.openstreetmap.de/ticket/22967#comment:1

Meanwhile, OP has mentioned on Telegram that they've been able to fix their query by using out meta;

[mmd]

So my follow-up questions would be:

• why "download from OSM along this track" still uses the OSM API, even though I have enabled object download using Overpass API.
• why does the process abort with an error message when it hits rate limiting, rather than waiting some time. Would this be an option?

[mrgenie]

Replying to mmd:

So my follow-up questions would be:

• why "download from OSM along this track" still uses the OSM API, even though I have enabled object download using Overpass API.

I have no idea about the programming behind the "download along this track" plugin but it seems limited to hard-coded OSM API, and ignoring the core application settings.

• why does the process abort with an error message when it hits rate limiting, rather than waiting some time. Would this be an option?

yes most certainly! My main issue I have is the manually restarting to download a dozen times until I finally got everything.
if it would download everything properly - just gotta wait a while - fine by me I can do other things meanwhile.
Right now I can't do anything else because I need to manually try again endlessly until I got it all.

So if JOSM would just "wait" when it receives this error code and try and again until it got it - then for me it's okay, I'll keep myself busy with other things.

[skyper]

Replying to mmd:

So my follow-up questions would be:

• why "download from OSM along this track" still uses the OSM API, even though I have enabled object download using Overpass API.

Same for me. Guess that was not adjusted when the code was copied from the plugin and once the plugin was integrated into core.

• why does the process abort with an error message when it hits rate limiting, rather than waiting some time. Would this be an option?

See #21563 for another problem with the limit. Though overpass has also a limit as far as I know. Do we now how long we have to wait? This might be better dealt in another ticket, I guess.

[mmd]

See #21563 for another problem with the limit. Though overpass has also a limit as far as I know. Do we now how long we have to wait?

Right, both the OSM API and Overpass API are rate limited. Overpass API has some /api/status page, which - although not machine readable - gives you some hints on the expected wait time. Note that there's no guarantee that the server answering /api/status is the same which handles a subsequent /api/interpreter call. osmnx spent quite some effort on this topic (see ​https://github.com/gboeing/osmnx/pull/699, and probably a few follow up ones).

Now with the OSM API, I see that we're only sending some generic error message, which doesn't include any wait time, or any other clues which might be helpful for JOSM: ​https://github.com/zerebubuth/openstreetmap-cgimap/blob/master/src/process_request.cpp#L518-L522

When the rate limiting check was implemented many years ago by zerebubuth, this wasn't really a topic which has been considered. Nowadays, we might think of adding some HTTP headers like described in ​https://www.ietf.org/archive/id/draft-polli-ratelimit-headers-02.html and expose those details to the outside world.

Maybe it would be a good idea to improve rate limiting visibility in a separate issue on ​https://github.com/zerebubuth/openstreetmap-cgimap/issues