JOSM-Plugins git mirror

[stoecker]

Does somebody know who operated the JOSM plugins mirror? It is out of sync and I'd like to replace it with upload from JOSM. I'd need the base repo files for this containing the SVN info.

Checking the server code I planned to migrate plugins as well for #20126, but it wasn't done. Probably because I didn't know who runs it...

See also #20126, #9483, #6887.

[taylor.smock]

Right now I'm running the ​josm-plugins mirror. I'll check and see if there was anything special I did (beyond adding an authors file to convert from the svn committer to git committer).

[taylor.smock]

$ git svn fetch --authors-file=authors.txt
$ git svn rebase --authors-file=authors.txt
$ git commit -asvm 'tmp' # This is needed to a file in MS Streetside that ''refuses'' to have the "right" line endings on my machine. You might be able to ignore this step.
$ git rebase --continue 
$ git push -u origin HEAD~1:master

[stoecker]

Currently my code looks like

sub mirror
{
  for my $name ("josm")#, "josm-plugins")
  {
    chdir "$name";

    # Commit externals changes, if any
    system("git config user.name \"JOSM GitHub mirror\" >$logfile 2>&1");
    system("git config user.email mirror\@josm.openstreetmap.de >>$logfile 2>&1");

    # Pull changes from Subversion
    system("git checkout master >>$logfile 2>&1");
    system("git svn fetch >>$logfile 2>&1");
    system("git svn rebase >>$logfile 2>&1");

    # Push the mirror to GitHub
    #system("git remote add mirror git\@github.com:JOSM/$name.git >>$logfile 2>&1");

    # Push to our mirrors
    system("git push mirror master >>$logfile 2>&1");
  }
  chdir $home;
}

I assume we can use this, expanded by the authors stuff?

Do I need to have the svn-base repo as binary again or is there a chance to regenerate this on the server? Last time (see #20126) this didn't work.

P.S. I deleted the authors file after download.

[stoecker]

P.S. I find it really troublesome that after searching for at least an hour I was not able to find out in the GitHub interface who actually uploads the data...

[taylor.smock]

I assume we can use this, expanded by the authors stuff?

I would assume so. Fair warning: a new committer will cause issues.

P.S. I deleted the authors file after download.

Works for me. I'll send an updated one via email. I think most of the ones from that one were scrapped off of GitHub.

who actually uploads the data

​https://github.com/JOSM/josm-plugins/branches

[stoecker]

Replying to taylor.smock:

who actually uploads the data

​https://github.com/JOSM/josm-plugins/branches

OK. That means I'm blind. I read that page and wasn't able to separate push and commit. Because you're the author of many commits and I did assume you're not the operator I simply ignored what my eyes did tell me.

I would assume so. Fair warning: a new committer will cause issues.

Please explain?

[taylor.smock]

Because you're the author of many commits and I did assume you're not the operator

Fair assumption. I did rename the one that was maintained by someone else to original-svn, just in case. But it looks like it isn't telling us anything anymore.
​https://github.com/JOSM/josm-plugins/activity?ref=master is probably a better link (from three dots on right side of branch -> Activity)

Please explain?

When git svn is provided with an authors file, it expects that every author will have a mapping to a git committer. It can be something like foobar = foobar <foobar@hashcode>, but it has to exist.

[stoecker]

I'll have a look at the weekend. Steps to come from the svn.tar.xz to full repo are appreciated...

When git svn is provided with an authors file, it expects that every author will have a mapping to a git committer. It can be something like foobar = foobar <foobar@hashcode>, but it has to exist.

Ah. Means I need to sync the access and authors file. Should be doable.

[taylor.smock]

Steps to come from the svn.tar.xz to full repo are appreciated...

This should do it:

1. put contents of svn.tar.xz in cloned repo (e.g. tar xf svn.tar.xz in the root directory for the git repo)
2. echo '5026c5627f2cacfb2410505a869fd915211edf41' > .git/refs/remotes/git-svn
3. add the following to .git/config

   [svn-remote "svn"]
   	url = https://josm.openstreetmap.de/osmsvn/applications/editors/josm/plugins
   	fetch = :refs/remotes/git-svn
   

Altogether:

$ git clone git@github.com:JOSM/josm-plugins.git
$ cd josm-plugins
$ tar xf ~/Downloads/svn.tar.xz
$ echo '5026c5627f2cacfb2410505a869fd915211edf41' > .git/refs/remotes/git-svn
$ cat <<EOF >> .git/config
[svn-remote "svn"]
        url = https://josm.openstreetmap.de/osmsvn/applications/editors/josm/plugins
        fetch = :refs/remotes/git-svn
EOF

I'll see if I can find an easy plugin bug to fix to so you can make certain that git svn fetch --authors-file=authors.txt && git svn rebase --authors-file=authors.txt will work.

[stoecker]

P.S. ​https://github.com/JOSM/josm-plugins/pull/22 Whoever programmed the dependabot did a really good job ;-)

[taylor.smock]

Yep. I did force trigger a rebase, but that should happen automatically.

It will eventually notice that we haven't actually merged any of its PRs and stop, but I think I can re-enable it when that happens.

[stoecker]

Replying to taylor.smock:

Yep. I did force trigger a rebase, but that should happen automatically.

We'll see next time...

It will eventually notice that we haven't actually merged any of its PRs and stop, but I think I can re-enable it when that happens.

As far as I can see you need to manually disable it when unwanted. So I think our way of fixing the issues is fine for this system. And as the issues are closed as "fixed" I doubt it even knows that the PRs were not the reason for it.

[taylor.smock]

IIRC, it disabled itself on the JOSMPluginAction repo since I wasn't actually merging its PRs -- I was fixing the problems, but there was an additional step to rebuild everything I needed to do.

[taylor.smock]

Replying to taylor.smock:

I'll see if I can find an easy plugin bug to fix to so you can make certain that git svn fetch --authors-file=authors.txt && git svn rebase --authors-file=authors.txt will work.

Looks like the instructions work with r36342/osm. Let me know if there are problems.

[taylor.smock]

Alternative to updating the josm-plugins repo: ​https://github.com/tsmock/josm-osmsvn-svn2git/tree/master/applications/editors/josm/plugins

[stoecker]

Replying to taylor.smock:

Alternative to updating the josm-plugins repo: ​https://github.com/tsmock/josm-osmsvn-svn2git/tree/master/applications/editors/josm/plugins

Yeah I know. Haven't forgotten about that. OTOH what we currently have works and that means my motivation to change it isn't high. You probably know that by now about me ;-)