Track is not rendered

[GerdP]

What steps will reproduce the problem?

1. Make sure that the Elevation profile plugin is not installed
2. Import file TrailRunning_2024-03-30T10_06_54.fit (see #23603)

What is the expected result?

A track is rendered and JOSM zooms to it.

What happens instead?

Nothing is rendered, but JOSM seems to know the bbox as it zooms to the correct area.

Please provide any additional information below. Attach a screenshot if possible.

When I "Convert to data layer" that still doesn't render anything. Only when I run the validatar I see the track in the "Validation erroes" layer.
No idea where to look at to fix this.

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2024-04-09 18:11:42 +0200 (Tue, 09 Apr 2024)
Revision:19039
Build-Date:2024-04-10 01:31:01
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (19039 en_GB) Windows 10 64-Bit
OS Build number: Windows 10 Pro 2009 (19045)
Memory Usage: 677 MB / 1888 MB (268 MB allocated, but free)
Java version: 21.0.1+12-LTS, Azul Systems, Inc., OpenJDK 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1080 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→32×32, 32×32→32×32
System property file.encoding: UTF-8
System property sun.jnu.encoding: Cp1252
Locale info: en_GB
Numbers with default locale: 1234567890 -> 1234567890
VM arguments: [-Dicedtea-web.bin.location=%UserProfile%\AppData\Local\Programs\OpenWebStart\javaws, -Djava.util.Arrays.useLegacyMergeSort=true, --add-exports=jdk.deploy/com.sun.deploy.config=ALL-UNNAMED, --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED, --add-reads=java.naming=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.security.action=ALL-UNNAMED,java.desktop, --add-reads=java.base=ALL-UNNAMED,java.desktop, --add-exports=java.naming/com.sun.jndi.toolkit.url=ALL-UNNAMED,java.desktop, --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED, --add-exports=java.desktop/com.apple.eawt=ALL-UNNAMED, --add-exports=java.desktop/sun.awt=ALL-UNNAMED,java.desktop, --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED, --add-exports=java.base/sun.security.validator=ALL-UNNAMED,java.desktop, --add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED, --add-exports=java.base/sun.net.www.protocol.jar=ALL-UNNAMED,java.desktop, --add-exports=java.base/jdk.internal.util.jar=ALL-UNNAMED,java.desktop, --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED, --add-exports=java.base/com.sun.net.ssl.internal.ssl=ALL-UNNAMED,java.desktop, --add-exports=javafx.graphics/com.sun.javafx.application=ALL-UNNAMED, --add-modules=java.scripting,java.sql,javafx.controls,javafx.media,javafx.swing,javafx.web, --add-exports=java.desktop/sun.awt.X11=ALL-UNNAMED,java.desktop, --add-exports=java.desktop/sun.applet=ALL-UNNAMED,java.desktop,jdk.jsobject, --add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED,java.desktop, --add-opens=java.base/java.lang=ALL-UNNAMED, --add-exports=java.base/sun.security.util=ALL-UNNAMED,java.desktop, --add-exports=java.desktop/com.sun.imageio.spi=ALL-UNNAMED, --add-reads=java.desktop=ALL-UNNAMED,java.naming, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-exports=java.base/sun.security.x509=ALL-UNNAMED,java.desktop, --add-opens=java.desktop/javax.imageio.spi=ALL-UNNAMED, --add-exports=java.desktop/javax.jnlp=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.security.provider=ALL-UNNAMED,java.desktop, -Djava.security.manager=allow]

Plugins:
+ FIT (36192)
+ OpeningHoursEditor (36241)
+ RoadSigns (36226)
+ apache-commons (36176)
+ buildings_tools (36226)
+ comfort0 (36200)
+ ejml (36176)
+ geotools (36176)
+ jackson (36176)
+ jaxb (36118)
+ jts (36004)
+ o5m (36126)
+ opendata (36200)
+ pbf (36176)
+ poly (36126)
+ reltoolbox (36241)
+ reverter (36241)
+ undelete (36226)
+ utilsplugin2 (36241)

Tagging presets:
+ c:\josm\core\resources\data\defaultpresets.xml

Validator rules:
+ d:\java_tools\JOSM\mygeometry.mapcss
+ https://josm.openstreetmap.de/josmfile?page=Rules/GermanySpecific&zip=1
+ c:\josm\core\resources\data\validator\combinations.mapcss
+ c:\josm\core\resources\data\validator\geometry.mapcss

Last errors/warnings:
- 00000.427 W: extended font config - overriding 'filename.Malgun_Gothic=malgun.ttf' with 'MALGUN.TTF'
- 00000.428 W: extended font config - overriding 'filename.Myanmar_Text=mmrtext.ttf' with 'MMRTEXT.TTF'
- 00000.429 W: extended font config - overriding 'filename.Mongolian_Baiti=monbaiti.ttf' with 'MONBAITI.TTF'

[GerdP]

It seems the plugin creates waypoints with invalid lat/lon pairs like lat=42.802233660035874,lon=351.071388950139
This probably confuses the renderer. Not sure yet where this has to be fixed.

[GerdP]

possible fix

[GerdP]

@Taylor: I've not yet managed to configure my Eclipse so that it accepts the java 21 sources from the FIT plugin, so it's difficult for me to write a unit test. I can compile with ant though and the patch seems to fix the problem.
It seems also unclear how Garmin encodes negative lat values?

[taylor.smock]

I'll pull it down and check. The hard part is that I have a very limited set of sample areas.

I got a Garmin watch sometime after I wrote the plugin initially, and I'm at lat=39 lon=-108. I think the original test files I had were all positive lat/lon.

[taylor.smock]

@RicardoRodriguezFernandez: For the FIT file you uploaded in #23603 (attachment:ticket:23603:TrailRunning_2024-03-30T10_06_54.fit​​), should it be near ​Serantes?

Loading the file using GerdP's patch looks correct, but I'm not 100% certain on that.

[GerdP]

When I look at background image that is displayed for the track (with the patch applied) it seems very likely that the coordinates are correct as the track is close to existing roads. The program GPXsee from ​https://www.gpxsee.org which also supports the FIT format displays it at the same position.

[taylor.smock]

I concur. I mostly wanted to double check.

I did check some from my watch, and they look like they are in the right place.

Serantes is at 42.8068474, -8.9196399.
Grand Junction is at 39.0672568, -108.56448.

So we have only verified that the -360 works for longitude. I would suspect that it works for latitude as well, but we'll see if we get a bug report for that.

[GerdP]

Yes, probably ;)
Do you want me to commit the patch and release a new jar? I tried to write a unit test but to be honest I wouldn't know what to test exactly.

[taylor.smock]

Go ahead and commit the patch. I'll write the unit test using one of my personal tracks.

[GerdP]

In 36245/osm:

fix #23613: Track is not rendered

• negative longitude values are not encoded as expected, so subtract 360 if >= 180
• maybe negative latitude values still don't work

[GerdP]

In 36246/osm:

fix #23613: Track is not rendered

• dist

[taylor.smock]

In 36247/osm:

See #23613: Add non-regression test for longitudes < 0

Also add what appears to be a secondary (either enhanced or degraded) elevation field.

[RicardoRodriguezFernandez]

Replying to GerdP:

In 36246/osm:

fix #23613: Track is not rendered

• dist

Hi! I'm afraid I'm late to the party! I'm still far for being able to contribute to the patching process, but I will keep learning. The only answer I could have contributed is YES, the track is close to Serantes.

Must we expect that the track will be now correctly rendered? Thanks!

[taylor.smock]

Hrm. Out of 1393 records in my FIT track, we only recognize 170. But it looks like the bulk of the remaining (~1100) don't have GPS information, so they aren't something we are interested in. Except to remove from the unknown record field so it is easier to see stuff we are interested in.

Anyway, I believe I found another elevation field (as noted in the above commit message).

Must we expect that the track will be now correctly rendered? Thanks!

Yes, it should be correctly rendered as soon as you update the plugin.

Please continue reporting bugs/issues with the plugin -- many of the conversion functions were reverse engineered (as in, "I know that the coordinates are changing, and they should be <foo> based off of <bar>, and the function should look like <math>").

So far, we haven't encountered any funky numbers (e.g. 1.2345678), but it is entirely possible that something, somewhere, requires multiplying by some insignificant number (e.g. 1.00000001).

[RicardoRodriguezFernandez]

Hi!

Replying to taylor.smock:

Yes, it should be correctly rendered as soon as you update the plugin.

It is not as easy as going to Preferences > Plugins > Update plugins yet, is it? At least here, no update is detected: version 36192 is installed here. Could you point me to the procedure? Thanks!

Please continue reporting bugs/issues with the plugin -- many of the conversion functions were reverse-engineered (as in, "I know that the coordinates are changing, and they should be <foo> based off of <bar>, and the function should look like <math>").

So, are you modifying the plugin by only looking at the FIT data in your/our own FIT files? Is there no description of Garmin's standards?

[skyper]

Replying to RicardoRodriguezFernandez:

Replying to taylor.smock:

Yes, it should be correctly rendered as soon as you update the plugin.

It is not as easy as going to Preferences > Plugins > Update plugins yet, is it? At least here, no update is detected: version 36192 is installed here. Could you point me to the procedure? Thanks!

As far as I remember you need to manually download the plugin list again (most left button) and then the update button should work. Take a look at the plugin's version number in the list. It should be 36248.

[GerdP]

Is there no description of Garmin's standards?

Typically not. Garmin also keeps the structure of the *.img files secret.

[taylor.smock]

It is not as easy as going to Preferences > Plugins > Update plugins yet, is it? At least here, no update is detected: version 36192 is installed here. Could you point me to the procedure? Thanks!

It can take up to 10 minutes for the job to fire and update the plugin list. At least for the ones that are not in source:osm/applications/editors/josm/dist . I think the same job also updates the information in plugin, but I could be wrong.

So, are you modifying the plugin by only looking at the FIT data in your/our own FIT files?

Pretty much. There is some more information available under license, but I didn't want to sign it (I think most of the additional information has to do with what each field is).

Is there no description of Garmin's standards?

The FIT format is documented and available without having to agree to a license (see ​https://developer.garmin.com/fit/protocol/ ). Unfortunately, what each record ("global message") and field is for is not documented for those who have not agreed to a license, so it takes a bit of guess and check, assuming you have some idea of what the values you are looking for are supposed to be.

EDIT: As a specific example, latitude, longitude, and elevation information were all reverse engineered after I wrote the basic FIT parser. IIRC, I used a FIT file that richlv provided for that.

[RicardoRodriguezFernandez]

Replying to skyper:

Replying to RicardoRodriguezFernandez:

Replying to taylor.smock:

Yes, it should be correctly rendered as soon as you update the plugin.

It is not as easy as going to Preferences > Plugins > Update plugins yet, is it? At least here, no update is detected: version 36192 is installed here. Could you point me to the procedure? Thanks!

As far as I remember you need to manually download the plugin list again (most left button) and then the update button should work. Take a look at the plugin's version number in the list. It should be 36248.

Done! It worked early today. All systems (macOS, Windows and Ubuntu) updated to FIT plugin 36248. I opened several *.fit files and all succeed: perfect rendering and positioning!

Thanks!

[RicardoRodriguezFernandez]

Replying to taylor.smock:

The FIT format is documented and available without having to agree to a license (see ​https://developer.garmin.com/fit/protocol/ ). Unfortunately, what each record ("global message") and field is for is not documented for those who have not agreed to a license, so it takes a bit of guess and check, assuming you have some idea of what the values you are looking for are supposed to be.

Why do we (OSM/JOSM) not agree with the Garmin FIT SDK license?

[taylor.smock]

Why do we (OSM/JOSM) not agree with the Garmin FIT SDK license?

I (the original author of the FIT plugin) didn't feel like reading and understanding the FIT SDK ​license. A quick scan-through of the license showed issues. Example:

[...] limited license to use the Licensed Technology for Licensee's internal business purposes, [...]
Licensee shall not, and shall not permit any third party to, directly or indirectly [...]

Those both seem to be incompatible with the GPL.