Opened 3 weeks ago
Last modified 2 weeks ago
#23875 new defect
Deploy pom.xml files to nexus
Reported by: | taylor.smock | Owned by: | team |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | unspecified | Version: | |
Keywords: | maven jenkins | Cc: | stoecker, floscher |
Description
@stoecker: Do you mind if I add a maven target to jenkins to deploy the pom files? It looks like I need to do that in order for the plugins (jenkins output) to use the same dependency versions as JOSM when we do a rebuild (as an example, for awhile apache-commons was using an older version of a dependency as compared to core -- that dependency is stripped down for core, apache-commons has the whole dependency).
As an alternative, I can copy the shared configuration into the plugin root pom. Or keep the dual ivy/maven dependency files.
Caveats:
I've been using 1.0-SNAPSHOT
in the pom files instead of SNAPSHOT
-- maven gives a warning without a number
[WARNING] [WARNING] Some problems were encountered while building the effective model for org.openstreetmap.josm:josm:jar:SNAPSHOT [WARNING] 'version' uses an unsupported snapshot version format, should be '*-SNAPSHOT' instead. @ org.openstreetmap.josm:josm-parent:SNAPSHOT, /Users/tsmock/workspace/josm/core/nodist/pom.xml, line 7, column 14 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING]
This means that we will be uploading SNAPSHOT
and 1.0-SNAPSHOT
builds for the foreseeable future.
The pom changes in core will be as follows:
-
nodist/pom.xml
diff --git a/nodist/pom.xml b/nodist/pom.xml
a b 39 39 <url>https://josm.openstreetmap.de/nexus/content/repositories/public/</url> 40 40 </pluginRepository> 41 41 </pluginRepositories> 42 <distributionManagement> 43 <snapshotRepository> 44 <id>josm-snapshots</id> 45 <name>JOSM snapshots</name> 46 <url>https://josm.openstreetmap.de/nexus/content/repositories/snapshots</url> 47 </snapshotRepository> 48 </distributionManagement> 42 49 <build> 43 50 <pluginManagement> 44 51 <plugins>
If we decide to use SNAPSHOT
instead of 1.0-SNAPSHOT
going forward, the snapshotRepository
element will instead be repository
.
Parameters I intend to set for Invoke top-level Maven targets
:
- Goals:
deploy
- Properties:
maven.test.skip=true
-- the tests are currently done by ant, and I don't think we want to double the CI runtime.skipTests
Attachments (0)
Change History (12)
comment:1 by , 3 weeks ago
comment:2 by , 3 weeks ago
Cc: | added |
---|
If I understand the snapshot topic correctly you want to publish something twice due to name issues?
More like version number issues. I don't know how we are currently publishing the JOSM snapshot to the server (probably a cron job or something). @Don-vip probably set that up.
If it was name issues, there is a standard way to do a relocation of the package.
The major problem with changing the versioning to be compliant with what maven expects and warns will eventually stop working (and is thus the standard) is that our downstream consumers (like the josm gradle plugin) may expect the non-standard SNAPSHOT
version. I have no clue if that is the case or not though. @floscher might have an idea.
I'd rather drop the old name and use the new one. [...] Currently my daily backup is already 26GB and the weekly at 84GB
Unless JOSM experiences a massive increase in size, I wouldn't anticipate that the backup size would increase significantly (call it 20mb extra/day, the SNAPSHOTS directory makes it look like it keeps ~1 month of snapshots, which comes to 0.6GB extra; if you round up for extra builds, that is 1GB extra).
Most of that 26GB is probably going to be from svn (josm svn is ~5.3GB, plugins svn is ~12.5GB, at least based off of my local full-copy mirrors).
comment:3 by , 3 weeks ago
More like version number issues. I don't know how we are currently publishing the JOSM snapshot to the server (probably a cron job or something). @Don-vip probably set that up.
Ah. That's two mvn deploy:deploy-file
in the daily Makefile.
mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${LATEST} -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${LATEST_SOURCES} -Djavadoc=${DOWNLOAD}/josm-latest-javadoc.jar mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm-unittest -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${DOWNLOAD}/josm-unittest.jar -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${DOWNLOAD}/josm-unittest-sources.jar
comment:4 by , 3 weeks ago
OK. I'd give floscher a chance to respond before we kill mvn deploy:deploy-file
in the daily Makefile, but absent objections, I can go ahead and make the changes in Jenkins.
comment:5 by , 3 weeks ago
Well, the release files you only get from the makefile. Jenkins doesn't build the releases.
comment:6 by , 3 weeks ago
Jenkins doesn't build the releases
If you are talking about josm-tested|latest
, this is correct
If you are talking about redistributable jar
files, this is not correct (one of the tasks called runs dist
), and it doesn't matter anyway since maven will do a build and package itself.
EDIT: Or I am misunderstanding something.
Realistically our maven stuff is really only useful for building/developing plugins.
comment:7 by , 3 weeks ago
Well, the above two lines are for josm-latest. That's the only signed release. Everything else (unsigned) already comes from Jenkins, but there are no "releases" there (i.e. no signing).
It's hard with all this CI to don't mix up stuff. :-)
comment:8 by , 3 weeks ago
That's the only signed release
I forgot about that.
It means I can't do it in jenkins. :(
With the patch, we should be able to use mvn deploy:deploy
and mvn deploy:deploy -Dversion=SNAPSHOT
instead of what we are currently doing, e.g.:
- mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${LATEST} -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${LATEST_SOURCES} -Djavadoc=${DOWNLOAD}/josm-latest-javadoc.jar - mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm-unittest -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${DOWNLOAD}/josm-unittest.jar -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${DOWNLOAD}/josm-unittest-sources.jar + mvn deploy:deploy -Dversion=SNAPSHOT + mvn deploy:deploy
I'll have to look into maven-jarsigner-plugin for that though. From the CLI, it looks like mvn -Dmaven.test.skip=true -DskipTests -Djarsigner.storepass=changeit -Djarsigner.keystore=./cert -Djarsigner.alias=josm package jarsigner:sign
should do it. That could probably be a little shorter, but that depends upon setting files.
comment:11 by , 2 weeks ago
For me, both version formats would be acceptable, no objections from me.
Regarding Gradle I worked around it, because it doesn't recognize the SNAPSHOT
version as possibly changing. See this issue: https://github.com/gradle/gradle/issues/5613 . At the time it seemed to be accepted to just use SNAPSHOT
by itself as a version number in Maven, but AFAIK it was and still is not by Gradle.
comment:12 by , 2 weeks ago
As noted in comment:10, I've switched our pom in source to use SNAPSHOT
instead of 1.0-SNAPSHOT
since it made life easier there -- I'm having to do some file manipulation in the CI to change the relative paths to core to be "right". I can remove those once we start publishing the poms.
As far as releasing 1.0-SNAPSHOT
versions, I think that mostly depends upon stoecker changing the version in the daily makefile. And hopefully switching from deploy-file
to deploy
.
Actually I understand only half of what you have written. Jenkins and maven are essentially a non-critical infrastructure for me (yes, yes not totally true anymore :-).
So when you want to change the way they work I have (probably? :-) no objections.
If I understand the snapshot topic correctly you want to publish something twice due to name issues? I'd rather drop the old name and use the new one. As said, it's no official external service of JOSM. While the server has enough space any additional data which is not cleared will increase my backups. I'd be happy if that does not happen. Currently my daily backup is already 26GB and the weekly at 84GB.