Plugin MapWithAI initialization may wait forever

[marcello@…]

If the initialization of the MapWithAILayerInfo singleton fails, the listeners will never get called and the plugin will wait forever at:

​https://github.com/JOSM/MapWithAI/blob/4af7ac225ff2ecf8428a011cdfbbdf06cf5e079a/src/main/java/org/openstreetmap/josm/plugins/mapwithai/data/mapwithai/MapWithAILayerInfo.java#L103C1-L103C1

This causes the integrationTest suite to never complete. The @Timeout on the test class
PluginHandlerTestIT has no effect.

I don't know why the plugin is failing, probably my fault, but it should never fail and lock up the whole show.

[taylor.smock]

@marcello: Did you get a thread dump and/or a stack trace? Or even console logs?

We can try adding threadMode = Timeout.ThreadMode.SEPARATE_THREAD to the @Timeout annotation so it doesn't hang.

[marcello@…]

Gradle is just sitting there spinning the clock:

./gradlew integrationTest --tests PluginHandlerTestIT
...
<============-> 93% EXECUTING [5m 6s]
> :integrationTest > 0 tests completed
> :integrationTest > Executing test org.openstreetmap.josm.plugins.PluginHandlerTestIT

I set the @Timeout to
@Timeout(value = 2, unit = TimeUnit.MINUTES,threadMode = Timeout.ThreadMode.SEPARATE_THREAD) so this doesn't work.

I guess I can get a stack trace.

[marcello@…]

stack traces of all running threads, search for "mapwithai"

[marcello@…]

That stacktrace taken at the end of PluginHandlerTestIT::testValidityOfAvailablePlugins()

[taylor.smock]

jstack would have been a better option -- it would tell me what threads are locking and/or waiting on which objects (e.g. waiting to re-lock in wait() <0x00000007802d1910> (a java.lang.ref.ReferenceQueue$Lock)).

Anyway, it does look like it is blocking in the MapWithAI plugin; we just have to figure out why.

ant test '-Ddefault-junit-includes=**/PluginHandlerTestIT.*' works (takes ~1.5 minutes on my machine).

You do know that we don't actually support the gradle scripts (they aren't in the repo, see #8269)? TBH, I'd like to move off of ant, if only because of IDE integration. I'd prefer maven since gradle has had way too many breaking changes over the past few years. And when maven has a breaking change, it is something that ends up getting broadcast far and wide.

[marcello@…]

I'm using my own gradle script. Was sick and tired of ant. Though gradle sucks plenty too.

I've loaded that plugin in JOSM now, JOSM starts but doesn't open mapviews any more. No error messages. The preferences > plugin list doesn't show either. No error messages. A bit disorienting.

I still don't know why the plugin init fails, but if it fails, it leaves the user pretty much in the dark.

[taylor.smock]

That is funny. It works for me. Here is a (trimmed) status report:

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2023-08-29 13:38:40 +0200 (Tue, 29 Aug 2023)
Revision:18822
Build-Date:2023-08-30 01:30:57
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (18822 en) Mac OS X 13.6.1
OS Build number: macOS 13.6.1 (22G313)
Memory Usage: 1252 MB / 4096 MB (565 MB allocated, but free)
Java version: 21.0.1+12-LTS, Azul Systems, Inc., OpenJDK 64-Bit Server VM
Look and Feel: com.apple.laf.AquaLookAndFeel
Screen: Display 188945233 1920×1080 (scaling 1.00×1.00) Display 188945231 1920×1080 (scaling 1.00×1.00) Display 69733382 1680×1050 (scaling 2.00×2.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→16×16, 32×32→32×32
Environment variable LANG: en_US.UTF-8
System property file.encoding: UTF-8
System property sun.jnu.encoding: UTF-8
Locale info: en_US
Numbers with default locale: 1234567890 -> 1234567890
Dataset consistency test: No problems found

Plugins:
+ apache-commons (36176)
+ mapwithai (816)
+ pmtiles (36125)
+ utilsplugin2 (36178)

[marcello@…]

It probably fails because of a change I made in my personal JOSM, its missing a function or so. Hard to say which without an error message.

[taylor.smock]

That makes stuff really hard to debug. Can you try to upstream your changes? Poke me on tickets (if they already exist and are ready).

I've been trying to avoid swallowing exceptions in JOSM/JOSM plugins (always log them). There is one exception -- the ProgressMonitorExecutor does not. Here is a local patch I have for that:

src/org/openstreetmap/josm/gui/progress/swing/ProgressMonitorExecutor.java

@@ -59 +59 @@
         }
         if (t != null) {
             Logging.error("Thread {0} raised {1}", Thread.currentThread().getName(), t);
+            Logging.error(t);
             BugReport.addSuppressedException(t);
         }
     }

[marcello@…]

It turns out it was accessing the global TaggingPresets (in MapWithAICategory::fromString), which is no longer global in my personal build because of test concurrency issues.

Expected: an error log with a stacktrace

Got: a lockup of the calling thread

BTW: somebody should look over this code ​MapWithAILayerInfo::getInstance

1. A partially constructed instance may escape for the second and all successive calls of getInstance while the first caller still waits for the construction to finish.

2. "Avoid a deadlock in the EDT." If the EDT is the first to call getInstance it must escape with a partially constructed instance.

3. If a non-EDT is the first to call getInstance, it faces a potential deadlock leaving instance in limbo forever. (This is what happened to me.)

[taylor.smock]

1. This isn't (usually) a problem. There is a bit of an ordering issue, in so far as if someone asks for the layers before it finishes initializing, there won't be layers. I don't think I've ever seen this failure mode.
2. This is (partially) fine; we don't need the layers until the user downloads something, and there is code that is called when the layers change, IIRC.
3. That should only be the case if the finish listener is never called, which means there should be an exception somewhere in the logs.

Anyway, I'm not going to try to fix the deadlock without figuring out why the finish code is never called. Which would also fix the deadlock.