[PATCH] AgPifoJ marks images (partially) incorrecly on the map

[TAA]

There was recently a problem with AgPifoJ, crashing when loading images. The latest version has fixed this, but now there is a problem with the placement of markers on the map:

when stopping and taking several photos at a spot sometimes the first photo may be shown at the location of the previous stop. This problem was never present in earlier releases of AgPifoJ so my quess is that something may have gone wrong with the latest patch?

[xeen]

Please provide photos and matching GPX track. Latest AgPifoJ version does a range match that is half the time difference between two GPX points. I.e. if you set your device to record your position every 30 seconds, AgPifoJ looks for a matching point +-15 seconds from the photo time. Is that the problem?

[TAA]

I am doing a track point every 10 meters, obviously the difference between time stamps could range from less than a second (doing 40 km/h on a bike) to, say, 30 seconds when stopping at a crossroads to take photos of the signs before moving again.

This was never a problem to AgPifoJ up to the versions about a month ago. Also the built-in photo geotag thing seems to still work - but it has no zoom and it is difficult to say where the image actually is on the track.

So, is the origine of the problem here the distance based tracking? If there are options to play around with AgPifoJ some guidelines would be useful indeed.

TAA

[TAA]

Another point: I always thought the geotagging of photos was done as follows:

1) corrected EXIF time of photo is "exactly" the same as a track point -> the coordinates are those of that track point

2a) corrected EXIF time of photo t is between track points A and B. Lets say the time stamp of A is A.t and coordinates are A.x and A.y, for B they are (B.t; B.x; B.y). If now c = (t - A.t)/(B.t – A.t) , ranging between 0 and 1, the geocoordinates of the photo would be

x = A.x + c(B.x – A.x) and

y = A.y + c(B.y – B.x).

This would assume a straight line and constant speed between points A and B, which should be an acceptable assumption in many cases. Should one seek to be more precise, the speeds at A and B could be also used, assuming a constant acceleration/deceleration. Now 2) would be:

2b) corrected EXIF time of photo t is between track points A and B. Lets say the time stamp of A is A.t and coordinates are A.x and A.y, for B they are (B.t; B.x; B.y). If now c = (t - A.t)/(B.t – A.t) , ranging between 0 and 1. Furthermore, the speed at A is A.v and at B it is B.v. Now at the point of the photo the (estimated) speed would be v = A.v + c(B.v – A.v) and the average speed over that distance v_avg= (A.v + v)/2, i.e. v_avg = A.v + c(B.v – A.v)/2. Now the coordinates of the photo would be:

x = A.x + (t - A.t)v_avg = A.x + (t - A.t)(A.v +c(B.v – A.v)/2) and

y = = A.y + (t - A.t)v_avg = A.y + (t - A.t)(A.v +c(B.v – A.v)/2)

This later option is obviously more complicated as the speed should be converted into coordinate units per second.

[TAA]

Replying to xeen:

Please provide photos and matching GPX track. Latest AgPifoJ version does a range match that is half the time difference between two GPX points. I.e. if you set your device to record your position every 30 seconds, AgPifoJ looks for a matching point +-15 seconds from the photo time. Is that the problem?

I would have now a small example, but even that's over 8 megabytes zipped. The system won't allow to upload it, and resampling the images would lose the exif data. The example shows clearly how the plugin and the built-in function come up in one case with quite different coordinates.

I've looked at the matchPoints code, and I wonder why there are two while loops there? The first one gives the current track coordinate if the time of the photo is within an "interval", i.e. time difference between previous and current track point. I believe the problem is there when using distance based tracking: if the time difference between previous and current was long (moving slowly) and then the next track point is only a short time ahead (picking up speed again), the code gives the current coordinates to photos that in fact should be further on the track.

The second while loop (linear interpolation) should be - in my view - sufficient for all cases except when current and previous track points happen to have the same time for some peculiar reason: this would cause a division by zero. In all other cases it should be ok, I think.

[xeen]

As far as I understood the code, there is no interpolation done at all – at least I had perfectly valid exif timestamps and still the photo's wouldn't match. Adding a few seconds offset fixed this for some photos but broke it again for others. If I remember correctly I only added the range stuff, but didn't change anything else related to the matching. You seem to have a good grasp of the code, maybe you can work it into a patch? I'm short on time and I really had problems with the code (and probably will still have when looking into this).

You can sent your sample to stefan@… (I can downsize it so everyone can have a look. If your mail client does not support 8 MB attachments, try splitting it up with a zip archive).

[xeen]

Hmpf, it abbreviated my mail address. Here we go again, remove the spaces:

stefan @ mathphys.fsk.uni-heidelberg.de

[TAA]

Replying to xeen:

Hmpf, it abbreviated my mail address. Here we go again, remove the spaces:

stefan @ mathphys.fsk.uni-heidelberg.de

OK Stefan!

I will email the data once I get to my own PC. I could even suggest some changes to the code. However, I will need help with building the code, although I studied IT quite a bit it was decades ago when FORTRAN was the standard and Pascal was stuctural programmers' wet dream ;-)

[xeen]

Testcase with super-compressed photos. Haven't looked at the issue yet.

[TAA]

The problem can be seen with the last image, which is shown on the track some 4 trackpoints too early. I've done some comparison with distance based and time based track points, and indeed the problem seems to be with the distance based tracking. Results are much better when using a fixed time interval.

However, I think the code could be fixed rather easily by patching just two methods.

[xeen]

I've had a look at the patches you sent me, but for some reason they place the photos off the track. I'm not yet sure what causes this, but maybe I'll find out…

[TAA]

Replying to xeen:

I've had a look at the patches you sent me, but for some reason they place the photos off the track. I'm not yet sure what causes this, but maybe I'll find out…

The code that calculates (interpolates) the point between two track points was already there, it is:

                        dateImgLst.get(i).pos = new EastNorth(
                                prevWp.eastNorth.east() + ((curWp.eastNorth.east() - prevWp.eastNorth.east()) * (imgDate - prevDateWp)) / (curDateWp - prevDateWp),
                                prevWp.eastNorth.north() + ((curWp.eastNorth.north() - prevWp.eastNorth.north()) * (imgDate - prevDateWp)) / (curDateWp - prevDateWp));
                        dateImgLst.get(i).coor = Main.proj.eastNorth2latlon(dateImgLst.get(i).pos);

As I expect, the code never goes into this loop, and it may in practise completely untested code. I wonder whether the arguments are in correct order (the name of the method suggests they are) and they are scaled correctly?

[xeen]

I don't know about the "previous" version, but this code is executed with the patches applied. While testing with "startIndex" still in place, changing this part of the code to just use "curWp"'s position without any interpolation made the photos be on the track again.

I'll post all future mails to this bug, so everything stays in one place.

[TAA]

patch to AgPifoj to better match the photos with the track

[anonymous]

The attached patch should fix the problems relating espcially distance-based tracking in the AgPifoJ plugin. A big thank-you to xeen, who made it possible to test the code and get it right!

I have tested the plugin with rather big data sets (some 200 photos taken over a bike trip of 40 km), both using time and distance based gpx tracks. It agrees nicely with the results of the built-in function. It does tag photos between track points, and this is the new idea here, i.e. capability to interpolate between track points.

The speed estimate shown next to the photos should be now better, too.

Thanks to xeen I can post the patch, which is tested by two different people.

[stoecker]

In [o14916].

[taylor.smock]

@TAA/xeen: I'm currently working on converting the JOSM plugin repo to git (see #23286). Do you have a preferred name/email for that?