Opened 19 months ago
Last modified 18 months ago
#23218 new defect
Update JDK versions in Java-EarlyAccess-JOSM
Reported by: | taylor.smock | Owned by: | team |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | Core | Version: | |
Keywords: | java21 | Cc: | stoecker |
Description
See also #22037 (the job doesn't appear to be working).
Java 21 was recently released. Temurin hasn't updated yet (due to needing to sign new legal agreements for the TCK). OpenJDK has a build available though.
We should also move JDK17 out of Java-EarlyAccess-JOSM
and Java-EarlyAccess-JOSM-Plugins
-- we are shipping JOSM installers with Java 17 right now, so I don't think JDK17 would qualify for "EarlyAccess" under any definition at this point.
We should probably drop JDK18 and JDK19 from those jobs (since neither one is still supported).
If we want to keep JOSM
and JOSM-Plugins
at two jobs for time reasons, we should either drop JDK11 or move it to a different job. Unless enough movement has occurred that we can (finally) close #17858. :)
Linux Version | default JRE | EOL |
---|---|---|
Ubuntu 20.04 | 11 | 2025-04-23 |
Ubuntu 22.04 | 11 | 2027-04-21 |
Ubuntu 23.04 | 17 | 2024-01-20 |
Debian 10 | 11 | 2022-09-10 |
Debian 11 | 11 | 2024-06 |
Debian 12 | 17 | 2026-06 |
Attachments (0)
Change History (59)
comment:2 by , 19 months ago
Replying to stoecker:
Too much text, too little commands!
I'd rather assume that a discussion is warranted, especially when I don't know the reasoning behind some things.
For example, are the EarlyAccess
jobs for Java EA releases? Or just new Java versions that we don't want to have "official" support for? I assume the former, but I could see an argument for the latter.
Do this?
- Enable java 21
Definitely.
- Java-EarlyAccess-JOSM add 21 and drop all else
- Java-EarlyAccess-JOSM-Plugins add 21 and drop all else
Probably for both of these. I know we can drop JDK18/JDK19 since those are no longer supported releases. JDK17 should probably move to the non-EarlyAccess jobs. If we don't move JDK17 to the non-EarlyAccess jobs, it shouldn't be removed from the EarlyAccess jobs.
- JOSM drop 11
- JOSM-Plugins drop 11
This really depends. I think we can justify dropping it while we support Java 8 and test against Java 8, so long as we replace Java 8 with it when we bump the minimum Java version to 11 (if we don't skip Java 11). This comes down to "do we want to test all LTS versions or just the first and last supported LTS versions". I'd rather test all of them, but there is a time penalty. I've been playing around with a patch to decrease test times, which pretty much comes down to removing System.gc()
calls; this causes us to spend an enormous amount of time in stop-the-world gc -- >50% of the test time is due to this.
- JOSM add 17
- JOSM-Plugins add 17
Definitely.
comment:3 by , 19 months ago
I think early access is from the time when Vincent was actively involved in reporting java bugs. We also can drop that totally if nobody uses it anymore.
From the time frame to get rid of java 8 it seems java 11 should be skipped and next step should be 17.
comment:4 by , 18 months ago
I'm completely open to dropping the EarlyAccess jobs. About the only "nice" thing about running an EA job is (if the EA version is constantly updated) we know about changes we have to make earlier rather than latter. But then we have to keep the EA release current, which we haven't been doing.
comment:5 by , 18 months ago
Ok. I disabled Early access and switched Java 11 to Java 17 for JOSM and Plugins.
Remains to add java 20 and 21 and enabled these as well.
comment:6 by , 18 months ago
Ok. Enabled 21 for JOSM and Plugins. Updated the Java list (including EA22), documented better what to do ;-)
Should work now.
@taylor: If you want to still have real EA, then you need to update the install-jdk.properties file. I'd copy it to the svn in this case.
comment:7 by , 18 months ago
Seems for JDK21 3 rendering tests need to be adapted.
And why does JOSM-Integration still fail when "No test failed"?
@taylor: If you want the EA I think I can automate the file update.
comment:8 by , 18 months ago
Should work now.
Thank you.
Seems for JDK21 3 rendering tests need to be adapted.
I noticed this on local runs; I had been assuming it was due to running the tests on macOS.
And why does JOSM-Integration still fail when "No test failed"?
It looks like it is due to the check-plugins
ant task, and it looks like it will fail on any plugin that depends upon Java 9 or later. There are specific exclusions for http2 and javafx.
[as:check-signature] Checking unresolved references to /var/lib/jenkins/jobs/JOSM-Integration/workspace/jdk/JDK8/plugin-check/api.sig [as:check-signature] Ignoring the signatures from file to be checked: /var/lib/jenkins/jobs/JOSM-Integration/workspace/jdk/JDK8/plugin-check [as:check-signature] Bad class file /var/lib/jenkins/jobs/JOSM-Integration/workspace/jdk/JDK8/plugin-check/FIT.jar:org/openstreetmap/josm/plugins/fit/FitPlugin.class
Looking at the task, I don't know why we are not running it when on Java 11+. I'll see if I can figure out why.
comment:11 by , 18 months ago
Hmm, JOSM-Integration is running JDK8 as far as I know.
We could:
1) Add exclusions (based on the manifest instead of hardcoding?)
2) Switch to Java 17
3) Add Java 17
ATM I'd still prefer point 1.
If we want to check all plugins we could add a task which uses JDK17 and only checks the >9 plugins?
follow-up: 13 comment:12 by , 18 months ago
Right now I'm adding exclusions via hardcoding. I've got it working locally, and I was just about to commit it.
I'd like to do it based off of manifest. I think we would have to script it somehow though -- I don't think ant can parse the /plugins endpoint anyway.
The one problem I did run into while testing my changes is that at least one file usually had a download issue. I worked around that (for testing) by reusing the ../dist
directory
<loadresource property="plugins"> <url url="https://josm.openstreetmap.de/plugin"/> <filterchain> <tokenfilter> <replacestring from="https://josm.openstreetmap.de/osmsvn/applications/editors/josm" to=".."/> </tokenfilter>
comment:13 by , 18 months ago
Replying to taylor.smock:
Right now I'm adding exclusions via hardcoding. I've got it working locally, and I was just about to commit it.
I'd like to do it based off of manifest. I think we would have to script it somehow though -- I don't think ant can parse the /plugins endpoint anyway.
Hmm. A script which adapts the build.xml would be easy ;-) Maybe 5 lines in perl...
Buuuuut. Doesn't it already read the plugin ressource. Should it be possible to filter for parameters?
https://ant.apache.org/manual/Types/filterchain.html
The one problem I did run into while testing my changes is that at least one file usually had a download issue. I worked around that (for testing) by reusing the
../dist
directory
<loadresource property="plugins"> <url url="https://josm.openstreetmap.de/plugin"/> <filterchain> <tokenfilter> <replacestring from="https://josm.openstreetmap.de/osmsvn/applications/editors/josm" to=".."/> </tokenfilter>
That's probably not an issue when running locally on the server.
follow-up: 15 comment:14 by , 18 months ago
Buuuuut. Doesn't it already read the plugin ressource. Should it be possible to filter for parameters?
Yes, it does read the plugin resource. The problem is that it is reading it line-by-line.
As an example:
FIT.jar;https://josm.openstreetmap.de/osmsvn/applications/editors/josm/dist/FIT.jar -> Keep (does not match `^\t.*`) Plugin-Minimum-Java-Version: 21 -> Remove (does match `^\t.*`)
I could try to use StripLineBreaks
, but I suspect that won't work well. It looks like it will just merge everything into a single file. I'll check and see if \r\n\t+
works, but I wouldn't be surprised if it doesn't. And I have no clue how it is going to merge the lines ("FIT.jar Plugin-Minimum-Java-Version" or "FIT.jarPlugin-Minimum-Java-Version"; I suspect the former).
That's probably not an issue when running locally on the server.
I agree. I just put it in writing for future people wondering how to fix EOL errors with the jar files.
comment:15 by , 18 months ago
Replying to taylor.smock:
Buuuuut. Doesn't it already read the plugin ressource. Should it be possible to filter for parameters?
Yes, it does read the plugin resource. The problem is that it is reading it line-by-line.
As an example:
FIT.jar;https://josm.openstreetmap.de/osmsvn/applications/editors/josm/dist/FIT.jar -> Keep (does not match `^\t.*`) Plugin-Minimum-Java-Version: 21 -> Remove (does match `^\t.*`)I could try to use
StripLineBreaks
, but I suspect that won't work well. It looks like it will just merge everything into a single file. I'll check and see if\r\n\t+
works, but I wouldn't be surprised if it doesn't. And I have no clue how it is going to merge the lines ("FIT.jar Plugin-Minimum-Java-Version" or "FIT.jarPlugin-Minimum-Java-Version"; I suspect the former).
Ah, I thought it reads it as a resource file and presents the indented sections as arguments.
comment:16 by , 18 months ago
Ah, I thought it reads it as a resource file and presets the indented sections as arguments.
That would have been so much easier.
Looking at the implementation for StripLineBreaks, it will not do what I thought it would. It just checks to see if a character is in the values list, and if it is, it removes it. So \r\n\t+
would merge everything into a single line.
comment:18 by , 18 months ago
@stoecker: Can you check and see what version of ant
is installed on the server? Specifically, the one at /var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_Latest/bin/ant
.
I suspect we are using ant 1.10.13 or earlier, since ant 1.10.14 "will no longer use Java SecurityManager because it has been deprecated for removal and by default is disallowed to be set at runtime".
comment:19 by , 18 months ago
ant 1.10.7 in the system 1.10.11 in the specified dir. I'll see if I can upgrade it.
comment:21 by , 18 months ago
Ok. I disabled Early access and switched Java 11 to Java 17 for JOSM and Plugins.
Was this a complete removal of Java 11? The JOSM-Plugins
job is trying to use /opt/jdk-11/bin/javadoc
.
Specifically,
/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_Latest/bin/ant -Djdk=JDK21 ******** clean test checkstyle spotbugs -Djavadoc.executable=/opt/jdk-11/bin/javadoc -Djavadoc.link=https://docs.oracle.com/en/java/javase/11/docs/api/ -Dplugin.dist.dir=../dist -Djosm=/var/lib/jenkins/jobs/JOSM/configurations/axis-jdk/JDK21/lastSuccessful/archive/dist/josm-custom.jar -Djosm.ivy=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK21/ivy.xml -Djosm.ivysettings=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK21/ivysettings.xml -Djosm.test.build.dir=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK21/test/build
That leads to a failure, since the /opt/jdk-11/bin/javadoc
executable (apparently) does not exist.
With that said, -Djavadoc.executable
should probably be ${JAVA_HOME}/bin/javadoc
(assuming jenkins replaces variables)
I don't know if we also want to update the -Djavadoc.link
setting for 17 and 21 (it looks like we just have to change 11
to 17
or 21
).
comment:22 by , 18 months ago
Ah. Legacy Legacy Legacy.
It should now (hopefully) use the javadoc of the corresponding version.
comment:23 by , 18 months ago
I don't think I can change the link matching to the version. I simply did set it to 21 ;-)
comment:25 by , 18 months ago
... it gets greener each day. Integration is orange now as well...
I think I just fixed one of the remaining issues for Integration (some icons I made for "widely" used tags had non-integer viewBoxes). So hopefully green next run. :)
I don't think I can change the link matching to the version. I simply did set it to 21 ;-)
Works for me.
comment:27 by , 18 months ago
remove hardcoded plugin Java versions
I hope the ant side works. Only tested the server side ;-)
comment:29 by , 18 months ago
I really hate ant. It's an awful build system. checkstyle for pmtiles failed with a nonsense error message "Unable to process files". I wasn't really able to find out what the reason is. Google search suggests it somehow doesn't like the files, but no idea why. By accident I found that it failed due to an empty "JCSCachedTileLoaderJob.java" file. I removed this. Forgotten debug stuff?
comment:30 by , 18 months ago
Not quite. I had copied that file from JOSM (see #23112), and I thought I deleted it. I guess not; I instead deleted the contents of the file.
comment:31 by , 18 months ago
comment:33 by , 18 months ago
maven
and gradle
have much better IDE support (we could probably get rid of all of the IDE specific configuration files). Beyond that, yeah, not much better.
I do strongly dislike gradle
due to all the API breakage, but it is really easy to script stuff. I think you can do some scripting in maven
with a plugin, but that isn't something I've ever tried.
comment:35 by , 18 months ago
Fair enough. The IDE I use (IDEA) has some nice features I like (debugging, profiling, hot reloading of a method, going to a method declaration, the ability to run test classes or methods individually in bulk, and so on).
We also have external contributors occasionally, and supporting IDEs for those people does make sense. I'd like to have more external contributors to JOSM.
comment:36 by , 18 months ago
All green except Plugins JDK21 and again I'm not really sure what the issue is.
comment:37 by , 18 months ago
I was able to reproduce by running ant clean dist test
in the utilsplugin2 directory with Java 21. Here is the stacktrace:
/Users/tsmock/workspace/josm/plugins/build-common.xml:603: Forked test(s) failed with an exception at org.apache.tools.ant.taskdefs.optional.junitlauncher.confined.JUnitLauncherTask.forkTest(JUnitLauncherTask.java:322) at org.apache.tools.ant.taskdefs.optional.junitlauncher.confined.JUnitLauncherTask.execute(JUnitLauncherTask.java:117) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at java.base/java.util.Vector.forEach(Vector.java:1359) at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:67) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99) at org.apache.tools.ant.Task.perform(Task.java:350) at org.apache.tools.ant.Target.execute(Target.java:449) at org.apache.tools.ant.Target.performTasks(Target.java:470) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1401) at org.apache.tools.ant.Project.executeTarget(Project.java:1374) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1264) at org.apache.tools.ant.Main.runBuild(Main.java:818) at org.apache.tools.ant.Main.startAnt(Main.java:223) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)
I'll check and see if this is already fixed in the ant
junitlauncher
task, and if not, I'll file a ticket with them (and maybe a patch).
comment:39 by , 18 months ago
Everyday a new error ;-) Now FIT code seems to behave different on JAVA 21...
comment:40 by , 18 months ago
FIT code seems to behave different on JAVA 21...
FIT code is Java 21 only. :)
But the problem appears to be error_prone. I was looking into whether or not we could set up ivy to have different dependency versions for newer versions of Java.
Context: error prone 2.10 was the last version to support compiling with Java 8. error prone 2.20 was when they added support for Java 21. They are currently thinking about dropping support for compiling with Java 11. Using --release
or --source
and --target
javac
options will still work, but the compiler would need to be Java 11+.
comment:41 by , 18 months ago
How unsane is it to make error checking tools which don't support the still most often used java version? When I look at commercial software Java 8 is still nearly 100%.
comment:42 by , 18 months ago
I need to correct myself; I was looking at the build output. It looks like one of the FIT tests is failing as well.
How unsane is it to make error checking tools which don't support the still most often used java version? When I look at commercial software Java 8 is still nearly 100%.
- New jakarta EE versions only support Java 11+ (although a lot of the packages are still compiled for Java 8)
- Spring Boot only supports Java 17 or later
- Apache Spark is planning on dropping Java 8 and 11 support in the next major release
In short, it looks like this year is the year where major frameworks are dropping support for Java 8. And error prone technically still supports Java 8, but you need to run it with a javac from Java 11 or later.
comment:46 by , 18 months ago
I'm still working on fixing the JOSM-Plugins jenkins job. Which is fun, since it only runs once a day. I have no clue what ivy_settings.xml
file is borking things, since I don't know which plugin was being built when it errors out.
Something either has an ivy_settings.xml
file when it doesn't on my machine or there is something different in the server environment that I don't understand. I haven't been able to reproduce the issue in the JDK17 job ("build-common.xml:749: impossible to resolve dependencies:") with a clean checkout of JOSM code (as in rm -rf josm_clean && svn co https://josm.openstreetmap.de/osmsvn/applications/editors/josm josm_clean && cd josm_clean/plugins && ant clean dist test
).
$ cd josm_clean && find -name 'ivy*settings.xml' ./core/ivysettings.xml ./plugins/geotools/ivy_settings.xml ./plugins/lwjgl/ivy_settings.xml ./plugins/OsmInspectorPlugin/ivy_settings.xml
I know which plugins are the problem for the Java 8 job (see #21995 -- same stack trace), and I think I'll just exclude them from Java 8 builds.
- Mapillary
- Dxf-Import
- photo_geotagging
comment:48 by , 18 months ago
Thank you.
Now I just have to wait for JDK8 to fail and see what happens with JDK17/JDK21.
comment:49 by , 18 months ago
Still failing. :(
I've managed to reproduce locally using
JAVA_HOME="$(/usr/libexec/java_home -v 17)" ant clean test checkstyle spotbugs \ -Dplugin.dist.dir=/Users/tsmock/workspace/josm/dist \ -Djosm=/Users/tsmock/workspace/josm/core/dist/josm-custom.jar \ -Djosm.ivy=/Users/tsmock/workspace/josm/core/ivy.xml \ -Djosm.ivysettings=/Users/tsmock/workspace/josm/core/ivysettings.xml \ -Djosm.test.build.dir=/Users/tsmock/workspace/josm/core/test/build
Current commandline script (server):
$ /var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/Ant_Latest/bin/ant -Djdk=JDK17 ******** clean test checkstyle spotbugs \ -Djavadoc.executable=/opt/JDK17/bin/javadoc \ -Djavadoc.link=https://docs.oracle.com/en/java/javase/21/docs/api/ \ -Dplugin.dist.dir=../dist \ -Djosm=/var/lib/jenkins/jobs/JOSM/configurations/axis-jdk/JDK17/lastSuccessful/archive/dist/josm-custom.jar \ -Djosm.ivy=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK17/ivy.xml \ -Djosm.ivysettings=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK17/ivysettings.xml \ -Djosm.test.build.dir=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK17/test/build
I'm doing a local build right now with debug enabled. Hopefully I can figure out what is causing the issue today.
comment:51 by , 18 months ago
For Integration: I think the plugins which require version 11 plugins (geotools) should also require version 11.
comment:52 by , 18 months ago
I assume you are talking about setting the java.lang.version
property along with the plugin.minimum.java.version
. Realistically, they should probably be the same value for most plugins. Example:
<property name="java.lang.version" value="8" unless:set="plugin.minimum.java.version"/> <property name="java.lang.version" value="${plugin.minimum.java.version}" if:set="plugin.minimum.java.version"/>
I was actually thinking about moving off of Java 8 for JOSM core -- if we do that, then we might as well compile all plugins for the same Java version as well. But meanwhile, since I'm not doing major feature work on the geotool plugins, I decided to leave the plugin's source code at Java 8. If I didn't, I'd have to open them up in Eclipse to properly update the Java SDK version.
comment:53 by , 18 months ago
No. I mean plugin.minimum.java.version. Integration fails because geotools cannot be loaded. That can only mean a plugin tries to load it and it should not, because the test is Java 8...
comment:54 by , 18 months ago
I thought geotools is java 11 now as the included classes need java 11?
comment:55 by , 18 months ago
I have geotools set up to be built with Java 11+. It is still compiled against Java 8 (so the classfiles are Java 8); the geotools libraries are Java 11+.
Anyway, looking at the integration test, I either missed updating the build.xml
for it or something happened and I didn't commit it.
org.opentest4j.AssertionFailedError: invalidManifestEntries: [] loadingExceptions: [ImportImagePlugin=java.lang.ClassNotFoundException: org.geotools.api.data.DataSourceException]
comment:59 by , 18 months ago
I took a look at why the Nexus-JOSM-Plugins
job was failing.
We either need to add -Djosm.ivysettings=/var/lib/jenkins/jobs/JOSM/workspace/jdk/JDK8/ivysettings.xml
or figure out why plugins are being built in that job.
/var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/JDK8/build-common.xml:749: settings file does not exist: /var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/core/ivysettings.xml
Default is ../core/ivysettings.xml
, which is why it is looking in jdk/core
.
Too much text, too little commands!
Do this?