Deploy pom.xml files to nexus

[taylor.smock]

@stoecker: Do you mind if I add a maven target to jenkins to deploy the pom files? It looks like I need to do that in order for the plugins (jenkins output) to use the same dependency versions as JOSM when we do a rebuild (as an example, for awhile apache-commons was using an older version of a dependency as compared to core -- that dependency is stripped down for core, apache-commons has the whole dependency).

As an alternative, I can copy the shared configuration into the plugin root pom. Or keep the dual ivy/maven dependency files.

Caveats:
I've been using 1.0-SNAPSHOT in the pom files instead of SNAPSHOT -- maven gives a warning without a number

[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.openstreetmap.josm:josm:jar:SNAPSHOT
[WARNING] 'version' uses an unsupported snapshot version format, should be '*-SNAPSHOT' instead. @ org.openstreetmap.josm:josm-parent:SNAPSHOT, /Users/tsmock/workspace/josm/core/nodist/pom.xml, line 7, column 14
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING] 

This means that we will be uploading SNAPSHOT and 1.0-SNAPSHOT builds for the foreseeable future.

The pom changes in core will be as follows:

nodist/pom.xml

@@ -39 +39 @@
             <url>https://josm.openstreetmap.de/nexus/content/repositories/public/</url>
         </pluginRepository>
     </pluginRepositories>
+    <distributionManagement>
+        <snapshotRepository>
+            <id>josm-snapshots</id>
+            <name>JOSM snapshots</name>
+            <url>https://josm.openstreetmap.de/nexus/content/repositories/snapshots</url>
+        </snapshotRepository>
+    </distributionManagement>
     <build>
         <pluginManagement>
             <plugins>

If we decide to use SNAPSHOT instead of 1.0-SNAPSHOT going forward, the snapshotRepository element will instead be repository.

Parameters I intend to set for Invoke top-level Maven targets:

• Goals: deploy
• Properties:
  • maven.test.skip=true -- the tests are currently done by ant, and I don't think we want to double the CI runtime.
  • skipTests

[stoecker]

Actually I understand only half of what you have written. Jenkins and maven are essentially a non-critical infrastructure for me (yes, yes not totally true anymore :-).

So when you want to change the way they work I have (probably? :-) no objections.

If I understand the snapshot topic correctly you want to publish something twice due to name issues? I'd rather drop the old name and use the new one. As said, it's no official external service of JOSM. While the server has enough space any additional data which is not cleared will increase my backups. I'd be happy if that does not happen. Currently my daily backup is already 26GB and the weekly at 84GB.

[taylor.smock]

If I understand the snapshot topic correctly you want to publish something twice due to name issues?

More like version number issues. I don't know how we are currently publishing the JOSM snapshot to the server (probably a cron job or something). @Don-vip probably set that up.
If it was name issues, there is a standard way to do a ​relocation of the package.

The major problem with changing the versioning to be compliant with what maven expects and warns will eventually stop working (and is thus the standard) is that our downstream consumers (like the josm gradle ​plugin) may expect the non-standard SNAPSHOT version. I have no clue if that is the case or not though. @floscher might have an idea.

I'd rather drop the old name and use the new one. [...] Currently my daily backup is already 26GB and the weekly at 84GB

Unless JOSM experiences a massive increase in size, I wouldn't anticipate that the backup size would increase significantly (call it 20mb extra/day, the SNAPSHOTS directory makes it look like it keeps ~1 month of snapshots, which comes to 0.6GB extra; if you round up for extra builds, that is 1GB extra).

Most of that 26GB is probably going to be from svn (josm svn is ~5.3GB, plugins svn is ~12.5GB, at least based off of my local full-copy mirrors).

[stoecker]

More like version number issues. I don't know how we are currently publishing the JOSM snapshot to the server (probably a cron job or something). @Don-vip probably set that up.

Ah. That's two mvn deploy:deploy-file in the daily Makefile.

mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${LATEST} -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${LATEST_SOURCES} -Djavadoc=${DOWNLOAD}/josm-latest-javadoc.jar
mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm-unittest -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${DOWNLOAD}/josm-unittest.jar -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${DOWNLOAD}/josm-unittest-sources.jar

[taylor.smock]

OK. I'd give floscher a chance to respond before we kill mvn deploy:deploy-file in the daily Makefile, but absent objections, I can go ahead and make the changes in Jenkins.

[stoecker]

Well, the release files you only get from the makefile. Jenkins doesn't build the releases.

[taylor.smock]

Jenkins doesn't build the releases

If you are talking about josm-tested|latest, this is correct
If you are talking about redistributable jar files, this is not correct (one of the tasks called runs dist), and it doesn't matter anyway since maven will do a build and package itself.
EDIT: Or I am misunderstanding something.

Realistically our maven stuff is really only useful for building/developing plugins.

[stoecker]

Well, the above two lines are for josm-latest. That's the only signed release. Everything else (unsigned) already comes from Jenkins, but there are no "releases" there (i.e. no signing).

It's hard with all this CI to don't mix up stuff. :-)

[taylor.smock]

That's the only signed release

I forgot about that.
It means I can't do it in jenkins. :(

With the patch, we should be able to use mvn deploy:deploy and mvn deploy:deploy -Dversion=SNAPSHOT instead of what we are currently doing, e.g.:

- mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${LATEST} -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${LATEST_SOURCES} -Djavadoc=${DOWNLOAD}/josm-latest-javadoc.jar
- mvn deploy:deploy-file --batch-mode -DgroupId=org.openstreetmap.josm -DartifactId=josm-unittest -Dversion=SNAPSHOT -Dpackaging=jar -Dfile=${DOWNLOAD}/josm-unittest.jar -DrepositoryId=nexus -Durl=https://josm.openstreetmap.de/nexus/content/repositories/snapshots -Dsources=${DOWNLOAD}/josm-unittest-sources.jar
+ mvn deploy:deploy -Dversion=SNAPSHOT
+ mvn deploy:deploy

I'll have to look into ​maven-jarsigner-plugin for that though. From the ​CLI, it looks like mvn -Dmaven.test.skip=true -DskipTests -Djarsigner.storepass=changeit -Djarsigner.keystore=./cert -Djarsigner.alias=josm package jarsigner:sign should do it. That could probably be a little shorter, but that depends upon setting files.

[taylor.smock]

In 36325/osm:

See #23875: Use SNAPSHOT instead of 1.0-SNAPSHOT since that is what our scripts currently publish

This should fix the Jenkins build for plugins.

Note: Maven warns that this may break in the future, in which case we'll have to
change our processes.

[taylor.smock]

In 19206/josm:

See #23875: Use SNAPSHOT instead of 1.0-SNAPSHOT since that is what our scripts currently publish

This should fix the Jenkins build for plugins.

Note: Maven warns that this may break in the future, in which case we'll have to
change our processes.

[floscher]

For me, both version formats would be acceptable, no objections from me.

Regarding Gradle I worked around it, because it doesn't recognize the SNAPSHOT version as possibly changing. See this issue: ​https://github.com/gradle/gradle/issues/5613 . At the time it seemed to be accepted to just use SNAPSHOT by itself as a version number in Maven, but AFAIK it was and still is not by Gradle.

[taylor.smock]

As noted in comment:10, I've switched our pom in source to use SNAPSHOT instead of 1.0-SNAPSHOT since it made life easier there -- I'm having to do some file manipulation in the CI to change the relative paths to core to be "right". I can remove those once we start publishing the poms.

As far as releasing 1.0-SNAPSHOT versions, I think that mostly depends upon stoecker changing the version in the daily makefile. And hopefully switching from deploy-file to deploy.