Server update

[stoecker]

New server is there and installation started

• created script setupsystem.sh containing the initialisation stuff from README and did these steps
• copied full backup-file
• TODO: Finish description how to install from backup in README
• TODO: a handful setup steps are missing yet.

@Vincent:

• SSH keys changed
• When you change something (i.e. in README), changes should be done in parallel on old system
  • Final step will be backup and full copy of josm home directory.

I'll continue tomorrow.

[Don-vip]

Yay! Thank you for taking care of it :) Let me know if you face problems with things I would have done on the old server.

[stoecker]

@Vincent:
Did most of the stuff. Website is up.

Lots of installation and config instructions moved into "update" and "setupsystem.sh" scripts.

Would be fine if you could have a look at the remaining issues.

When possible add them to the two scripts for automated handling.

NOTE: for "update" some changes are pending. That's wanted - the current config on new server is updated already (mostly :-).

Coverity install instructions don't work.

[stoecker]

ImageryCompare also hangs. Don't know why. I already created a dummy config and did set java-8 usage.

[Don-vip]

Replying to stoecker:

@Vincent:
Did most of the stuff. Website is up.

I can't access ​https://<ipv4>. I get error message "TracError: IOError: [Errno 13] Permission denied: u'/home/josm/trac/log/trac.log'". Is it expected?

EDIT: I changed logfile owner to josm:www-data as on current server and it works.

[Don-vip]

For ImageryCompare the issue is like the last migration:

$ groovy -v
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/usr/share/groovy/lib/groovy-2.4.15.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Groovy Version: 2.4.15 JVM: 10.0.2 Vendor: Oracle Corporation OS: Linux

$ ll /usr/lib/jvm/default-java
lrwxrwxrwx 1 root root 25 Apr  8  2018 /usr/lib/jvm/default-java -> java-1.11.0-openjdk-amd64/

/usr/lib/jvm/default-java must point to a JDK8 for Groovy to work.

[Don-vip]

Done. Is there another issue?

[stoecker]

Ah, seems my rsync call destroyed user permissions. Have to check the call :-)

/usr/lib/jvm/default-java must point to a JDK8 for Groovy to work.

Added to setupsystem.sh

Done. Is there another issue?

Can you check if nexus, sonar, jenkins and all the CI stuff work as expected. I synced whole jenkins stuff to old server, the others are fresh installed.

[Don-vip]

Replying to stoecker:

I synced whole jenkins stuff to old server

Many job definitions are missing (Java-EarlyAccess-Update-JDK, plus those related to Nexus and Sonar)

Sonar doesn't work, I'm looking at it.

[stoecker]

Replying to Don-vip:

Replying to stoecker:

I synced whole jenkins stuff to old server

Many job definitions are missing (Java-EarlyAccess-Update-JDK, plus those related to Nexus and Sonar)

Seems a restart of jenkins helped :-)

Sonar doesn't work, I'm looking at it.

Thanks.

[Don-vip]

Replying to stoecker:

Sonar doesn't work, I'm looking at it.

Thanks.

This isn't the correct version. Version installed is 7.1 (old), but we're running 7.5. The database is incompatible.

EDIT: the install script must be rewritten for Sonar. ​https://downloads.sourceforge.net/project/sonar-pkg is no longer maintained and Sonar must be now installed manually as per ​documentation.

[stoecker]

Can you do so? I'm done for today and have for sure no time tomorrow.

[Don-vip]

I just did it but my modifications in setupsystem.sh are gone. Did you just overwrite the file?

EDIT: wrong server...

[Don-vip]

su - sonar -c "/opt/sonar/bin/linux-x86-64/sonar.sh start" works but service sonar start does not. I don't get it.

[stoecker]

Plan: Switch server tomorrow evening. Remaining issues will be resolved in life system later ;-)

[GerdP]

Just a question: Do you plan to switch the server before releasing next JOSM tested version? IMHO it would be good to not do it in parallel, instead it would be good to do it right between two releases. Just in case something goes wrong we would start hunting errors in the wrong place.

[stoecker]

Replying to GerdP:

Just a question: Do you plan to switch the server before releasing next JOSM tested version? IMHO it would be good to not do it in parallel, instead it would be good to do it right between two releases. Just in case something goes wrong we would start hunting errors in the wrong place.

Then release has to wait. Server move is due till Thursday.

[GerdP]

OK. Good luck!

[stoecker]

Server move done. Report any issues.

[GerdP]

Great! Not sure if this is relevant: The mail that was generated for this comment appeared in my spam folder.

[GerdP]

I think Sonar Qube shows outdated data for JOSM plugins. Should I wait before I submit a change to plugin poly?

[GerdP]

Sample:
https://josm.openstreetmap.de/sonar/project/issues?fileUuids=83923eb9-dd68-4e7e-b71d-cd3c01ea19ec&id=josm-plugins&resolved=false

[Klumbumbus]

The "trac ticket content column width" is narrower now. I think 1000px width images did fit within the yellow area before, e.g. in #14666.

[GerdP]

There is a new Warning on Sonar Qube: CSS files were not analyzed. Node.js version could not be detected using command: node -v

[Don-vip]

@dirk we cannot access any file located in /home/josm/www, including robots.txt. We face http 403 forbidden.

[Klumbumbus]

Trac error after commenting/editing a ticket:

Warnung: Die Änderung Änderung wurde gespeichert, aber es trat ein Fehler beim Senden von Benachrichtigungen auf: (451, '4.3.5 <xxx.xxx@…>: Recipient address rejected: Server configuration error')

[Don-vip]

I have changed the sponsor logo to FOSSGIS on home page. We should add a note to thank them in StartupPage too.

[Klumbumbus]

That was an evil typo in your last comment :D

[simon04]

The Trac footer reads as follows:

Server sponsored by Hetzner. Visit the Openstreetmap project at ​https://www.openstreetmap.org/

Hetzner → FOSSGIS? Openstreetmap → OpenStreetMap?

[stoecker]

The "trac ticket content column width" is narrower now. I think 1000px width images did fit within the yellow area before, e.g. in #14666.

@dirk we cannot access any file located in /home/josm/www, including robots.txt. We face http 403 forbidden.

Fixed. Copied the config from main apache config to JOSM site, so it is not overlooked.

Hetzner → FOSSGIS? Openstreetmap → OpenStreetMap?

Fixed.

The mail that was generated for this comment appeared in my spam folder.

Very likely this will be the case for some months → New IP :-(

@Vincent: In /var/tmp is a 100% backup of old server.

[stoecker]

@Vincent: Maven says: Failed to transfer file: ... Return code is: 401, ReasonPhrase: Unauthorized

451, '4.3.5 <​xxx.xxx@…>: Recipient address rejected: Server configuration error'

Report if it happens again. I'm not sure if I fixed this.

[stoecker]

@everybody at gmail. Please move blocked mails from Quarantine or spam folder to your inbox!

[GerdP]

Not sure if this is relevant here: With r14751 I've accidentially introduced an empty line in file ignoretags.cfg. This causes a failure in the unit test TaginfoTestIT but https://josm.openstreetmap.de/jenkins/job/JOSM/5004/ reports "Kein Test fehlgeschlagen."
Maybe that test is not always executed?

[GerdP]

It seems I cannot distribute plugins:
Search for Failed to execute goal [32morg.apache.maven.plugins:maven-deploy-plugin in
https://josm.openstreetmap.de/jenkins/job/Nexus-JOSM-Plugins/731/console

[GerdP]

Maybe origin of comment:33 problem is this:

BUILD FAILED
/var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/JDK8/build.xml:47: The following error occurred while executing this line:
/var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/JDK8/build.xml:24: The following error occurred while executing this line:
/var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/JDK8/build-common.xml:214: /var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/JDK8/javafx/javadoc does not exist.

[GerdP]

This also looks like a new problem:

https://josm.openstreetmap.de/jenkins/job/JOSM-Integration/lastCompletedBuild/jdk=JDK8/testReport/org.openstreetmap.josm.plugins/PluginHandlerTestIT/testValidityOfAvailablePlugins/

2019-01-30 21:24:14,049 Timeout runner ERROR No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2

[Don-vip]

@Dirk since the server upgrade we get the following error in imagery integration test:

java.net.UnknownHostException: geoserver.dados.al.gov.br

But I have no problem to ​access it from my connection.

[Don-vip]

Nexus configuration wasn't done. I have shut down the service for security reasons and downloading the old config, it takes a while (the backup is 190Gb)

[stoecker]

Replying to Don-vip:

@Dirk since the server upgrade we get the following error in imagery integration test:

java.net.UnknownHostException: geoserver.dados.al.gov.br

But I have no problem to ​access it from my connection.

Hmm, w3m on commandline worked as expected a few minutes ago.

[simon04]

Building pt_assistant via Travis CI fails: ​https://travis-ci.org/JOSM/pt_assistant/jobs/487931560

* What went wrong:
Could not resolve all files for configuration ':compileClasspath'.
> Could not resolve org.openstreetmap.josm:josm:14382.
  Required by:
      project :
   > Could not resolve org.openstreetmap.josm:josm:14382.
      > Could not get resource 'https://josm.openstreetmap.de/nexus/content/repositories/releases/org/openstreetmap/josm/josm/14382/josm-14382.pom'.
         > Could not GET 'https://josm.openstreetmap.de/nexus/content/repositories/releases/org/openstreetmap/josm/josm/14382/josm-14382.pom'. Received status code 503 from server: Service Unavailable

[Don-vip]

Yes it took me a while to setup nexus back. It's live again.

[stoecker]

Is the "geoserver.dados.al.gov.br" issue gone? If yes, then the systemd DNS resolver was the reason. I now completely disabled it and use Hetzner DNSSEC enabled resolvers again.

[Don-vip]

Yes, it's working now. The current issues don't seem caused by our system.

[Don-vip]

The performance boost is huge. Jenkins JOSM job went from 28/30 min to 12/13 min:

[GerdP]

Yes, and also Trac is much faster now. Only svn update seems to be as slow as before. No idea if that depends on my machine?

[Don-vip]

@Dirk how did you install Jenkins? The new security update must be installed, but it is not proposed by apt, as on old server.

[stoecker]

Replying to Don-vip:

@Dirk how did you install Jenkins? The new security update must be installed, but it is not proposed by apt, as on old server.

As documented: See

setupsystem.sh

[stoecker]

Maybe you used ​https://pkg.jenkins.io/debian/ and not ​https://pkg.jenkins.io/debian-stable/ on old server?

[stoecker]

No, it seems simply the repo is not updated yet ;-)

[Don-vip]

Replying to stoecker:

No, it seems simply the repo is not updated yet ;-)

Indeed. It's very unusual, I hope the repo is still maintained, and will not vanish like SonarQube...

[Don-vip]

The jenkins update is now available :)