API calls blocking, cancel not working, JOSM hangs

[Ldp]

An unresponsive OSM API as of this time exposed the fact that the API capabilities call doesn't time out, thus hanging JOSM. I was about to upload when this happened, and I'm now even unable to save my layers.

GET http://api.openstreetmap.org/api/capabilities

[Ldp]

I hit this one again right now. When this happens, it causes a loss of data, because control never returns to JOSM, so one is unable to save any changes.

[Gubaer]

Did this happen when you uploaded or downloaded data?

In case of downloading: did you see a progress dialog an was there a Cancel button you could click? Did JOSM hang although you clicked cancel?

In case of uploading: did you try to upload an OSM file *before* having downloaded anything from the server?

Apologies for nagging questions, I'm trying to find out where this could happen, also, because it's difficult to reproduce. One problem is in ApiPreconditionCheckerHook​ but there it's quite complicated to turn the api.initialize() into an asynchronous, cancelable task. I'd rather like to fix this after the release and cleanup the pre-upload hooks in one go.

[Ldp]

These two cases happened on a fresh upload, without prior downloading.

That is: start JOSM, create a new layer, open a .osm file, merge stuff to the first layer, then upload that first layer.

It taught me to always save the working layers, before every upload.

In both of these cases the TCP session to the API server was established, but then there were no more reponses.

[Gubaer]

These two cases happened on a fresh upload, without prior downloading.

Ok, thanks, that's consistent with my understanding. That's due to the problem in ApiPreconditionCheckerHook. There, api.initialize() should run asynchronously and you should have a "Cancel" button to abort it if it hangs due to network problems.

Unfortunatelly, in order to fix this, we should refactor the way upload hooks are currently called.

• upload hooks shouldn't run on Swings EDT any more, but on an asynchronous thread.
• upload hooks should have access to a progress monitor in order to show progress info and to be cancellable. This has an impact on the plugin API interface.

I'd like to postpone this to the next iteration (post release 2010-01). Replacing keyword r-2010-01-blocker by r-2010-02.

[Fabi2]

After I reported a similar problem as #3970, I found this problem now. JOSM hangs due to the blocking network I/O used.

JOSM (via the JVM) opens a network connection which uses a blocking socket, which is the default unless changed with fnctl() on the socket file descriptor. Blocking I/O means, that the (Linux-)kernel will put the process which blindly does a write() or read() to the socket, are put to sleep until there are data to read or the last data which has to be written finally fits in the socket buffer. See "man 7 socket". Now go and pull the network plug during a read or withing a write with more date to be written... since SO_KEEPALIVE is off by default (afaik) the kernel blocks the application (JVM) forever.

The best solution ist to tell the JVM somehow to use event based asyncronous I/O on the socket, the second best is to use polling on a non-blocking socket, the third best is to interrupt the I/O-operation by a (timeout) signal.

[stoecker]

Ticket #5058 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #5138 has been marked as a duplicate of this ticket.

[Fabi2]

See #5404

[stoecker]

Ticket #4975 has been marked as a duplicate of this ticket.

[simon04]

Ticket #5369 has been marked as a duplicate of this ticket.

[simon04]

From #5369:

---

To reproduce:

1. Run a TCP server that never handles incoming connections, e.g. in Python:

   import SocketServer, time
   
   class ForeverHandler(SocketServer.BaseRequestHandler):
       def handle(self):
           time.sleep(10000)
   
   if __name__ == "__main__":
       HOST, PORT = "localhost", 9996
       server = SocketServer.TCPServer((HOST, PORT), ForeverHandler)
       server.serve_forever()
   

2. Change the OSM server URL in JOSM to http://localhost.com:9996/api
3. Do anything involving the API (e.g., download an bbox or fetch the history of an object)
4. Try to cancel the operation – fail.

---

The connection hangs somewhere inside Java's AbstractPlainSocketImpl, thus hard to change. I see the following two possibilities:

1. Switch to Apache's ​HttpClient library
2. Perform OsmConnections in separate Threads, but how to reliably kill the thread (see ​http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html)

What do you think?

[simon04]

A one line patch to (hopefully) fix this annoying issue. :-)

[Don-vip]

Nice !

[simon04]

In 5084/josm:

hopefully fix #4216 - API calls blocking, cancel not working, JOSM hangs