[PATCH] Import more data from NMEA GNSS tracks

[pyrog]

With Real Time Kinematic (RTK) more NMEA fields could be stored for each nodes, and used e.g. for colorise the track.

• The age in ​GGA message
• The full satellite count (for all constellations) in ​GSA message
• The standard deviation of horizontal error in ​GST message
• The differential reference station ID in ​GGA

Currently JOSM display the satellite count for the GPS constellation only.

To obtain it, count the non empty SVID in the ​GSA sentence.
21 in the following example (SVID 12,25,29,31,32 76,84 02,24,12,36,07,08,11,25 30,36,26,29,13,35)

$GNGGA,070832.25,4634.4821410,N,00545.0846823,E,4,12,0.59,548.778,M,47.205,M,1.3,0000*60
$GNGSA,A,3,12,25,29,31,32,,,,,,,,1.09,0.59,0.92,1*02
$GNGSA,A,3,76,84,,,,,,,,,,,1.09,0.59,0.92,2*00
$GNGSA,A,3,02,24,12,36,07,08,11,25,,,,,1.09,0.59,0.92,3*06
$GNGSA,A,3,30,36,26,29,13,35,,,,,,,1.09,0.59,0.92,4*06

The standard deviation horizontal error is the maximum of the standard deviation of latitude error or standard deviation or longitude error.
In this exemple, the standard deviation horizontal error is 0.019 meter (1.9 cm)

$GNGST,070832.50,32,0.053,0.037,48,0.018,0.019,0.035*4A

[pyrog]

Sample NMEA track in RTK

[StephaneP]

Now that Josm can parse the fix value inside nmea files (#20600) parsing the $GPGST $GNGST values would be a wonderful enhancement.

[StephaneP]

copy/paste of ​https://receiverhelp.trimble.com/alloy-gnss/en-us/NMEA-0183messages_GST.html

Position error statistics

An example of the GST message string is:

$GPGST,172814.0,0.006,0.023,0.020,273.6,0.023,0.020,0.031*6A

The Talker ID ($--) will vary depending on the satellite system used for the position solution:

    $GP - GPS only
    $GL - GLONASS only
    $GN - Combined

GST message fields
Field 	Meaning
0 	Message ID $GPGST
1 	UTC of position fix
2 	RMS value of the pseudorange residuals; includes carrier phase residuals during periods of RTK (float) and RTK (fixed) processing
3 	Error ellipse semi-major axis 1-sigma error, in meters
4 	Error ellipse semi-minor axis 1-sigma error, in meters
5 	Error ellipse orientation, degrees from true north
6 	Latitude 1-sigma error, in meters
7 	Longitude 1-sigma error, in meters
8 	Height 1-sigma error, in meters
9 	The checksum data, always begins with *

[stoecker]

I recommend to use QGIS for in depth analysis of NMEA data and not JOSM.

[StephaneP]

Replying to stoecker:

I recommend to use QGIS for in depth analysis of NMEA data and not JOSM.

The goal is not NMEA analysis, but viewing the position accuracy inside Josm, like I already do with .pos files. See this example:

A pos file (Quality (RTKlib only) + draw a circle from DOP value):

The same data in nmea (GPS Fix + draw a circle from DOP value):

The dop values inside a NMEA/GPX files are not very useful but the lat/long error ($G?GST) is when it is available.

(the dop values display in Josm for the .pos file are the lat/long error).

[stoecker]

DOP as well as GST error values are both only raw indications of the real accuracy. As an indicator they are good enough. Assuming you will get better results with the error estimates only gives you a false sense of accuracy. We already had cases when the error estimates have been in the 3 cm range, but the position accuracy was actually 50cm off (with a high end system). For the normal lower cost systems I expect that to be a much more common occurrence with higher errors (especially when using in a moving equipment).

So when you don't want to analyze the data itself the DOP should be enough.

[StephaneP]

DOP and GST are two very different things. DOP is only an indicator of the geometrical position of the received satellites.

What you see on the .pos exemple is the same as what you could see with the nmea file if the GST sentences were parsed. Do you really think it isn't more useful than the constant DOP circle size???

I'm confident that the accuracy result I have is ok. It was tested on various survey marker, it is used by many farmer, GIS guys, compared with high end products (trimble, topcon,...). It's a F9P from U-Blox connected on a GNSS base station from the ​centipede network. See this benchmark: ​http://gpspp.sakura.ne.jp/paper2005/IPNTJ_NEXTWG_202102.pdf
I don't say we never see some wrong accuracy values, but it's not a good argument to say "We prefer to display only the unuseful DOP values because sometime accuracy values could be wrong".

I can't explain your 50cm off without more details, but it looks like a wrong coordinate reference system. In France we have a 70cm offset between irtf@now and etrs89.

[stoecker]

It's my job to develop software applications for GNSS receivers since many years and yes for this application here I believe that the internal error calculations and the DOP values give you nearly the same result. Main influence is the number and distribution of available satellites (i.e. the DOP) and the used position fix.

Please don't understand me wrong: I don't mean the numerical values, but only the information if a position is to be trusted or not.

Contrary to what most people believe the error estimates from the GST do not tell you what the real error is, but only indicate the details of the position calculation matrix (i.e. an internal accuracy). This leaves out a lot of effects. BTW: Especially the F9P ublox in PPP mode has sometimes very high position offsets while still keeping good error estimates. Also in RTK mode we see wrong fixes. As said we've even seen 50cm of position for several minutes in real high end hardware and the vendor later changed the firmware to be more robust against such type of behaviour after they got our report.

GNSS systems are a really complex topic and it's even hard to explain to professional users what these numbers actually mean. I fear that when we display a 2cm value for a certain measurement in JOSM then people with such systems will overrule any other data and assume 2cm also means 2cm, but that is very often simply not true. Also if we go into these accuracy scales the topic of reference systems comes up which currently in OSM isn't addressed at all. Think of two users with an RTK system one with ETRS89, one with WGS84 or ITRF output who constantly change tracks because they KNOW their data is 2cm exact...

If JOSM already supports a quality indication for RTK lib we may implement it for NMEA as well, but as said it will give a false sense of accuracy.

[StephaneP]

patch for nmea GST sentences

[StephaneP]

Here is my try to parse the GST sentence and use the standard horizontal deviation to draw circle

• Nmea : add GST sentence parser
• Nmea : add test and nmea file for this test
• add constant for horizontal and vertical constant (PT_STD_HDEV and PT_STD_VDEV)
• use PT_STD_HDEV values if available for drawing circle on gpx layer
• RtkLib : put horizontal standard deviation to PT_STD_HDEV (and not to PT_STD_HDOP)
• RtkLib : put vertical deviation to PT_STD_VDEV

[StephaneP]

Circle from nmea gst horizontal standard deviation

[stoecker]

Some first comments based on short review:

• src/org/openstreetmap/josm/io/nmea/Sentence.java the comments are switched
• You cannot misuse the HDOP function to draw stddev circles

[StephaneP]

Replying to stoecker:

Some first comments based on short review:

• src/org/openstreetmap/josm/io/nmea/Sentence.java the comments are switched

Oh! You're right! I've fixed this in the updated patch

Replying to stoecker:

• You cannot misuse the HDOP function to draw stddev circles

I agree with you, and that's why the RtkLib part doesn't populate PT_STD_HDOP anymore.
The next step would be to add something in the Gui to display a choice for the circle (hdop, horizontal or vertical deviation, ...). But I'm a Java newbie. Perhaps I could add this later, after another more important feature I'd like to add.

[StephaneP]

New patch release with various fixes on checkstyle and links to catdb.org updated to gpsd.gitlab.io

[StephaneP]

Hi!

Is it possible to get some feedback on this patch?

[StephaneP]

One point that bothers me is the types of some variables:
The dop variables are a Float

currentwp.put(GpxConstants.PT_PDOP, Float.valueOf(accu));

but elsewhere in Josm, Floats aren't used very much, they're more like Doubles.
example in OsmDataLayer.java:

addDoubleIfPresent(wpt, n, gpxPrefix, GpxConstants.PT_HDOP, "gps:hdop");
addDoubleIfPresent(wpt, n, gpxPrefix, GpxConstants.PT_VDOP, "gps:vdop");
addDoubleIfPresent(wpt, n, gpxPrefix, GpxConstants.PT_PDOP, "gps:pdop");
addDoubleIfPresent(wpt, n, gpxPrefix, GpxConstants.PT_AGEOFDGPSDATA, "gps:ageofdgpsdata");

Would it be a problem if I changed these floats (hdop pdop vdop) to Double?

the "speed" value is another case. Why is it a String?

[StephaneP]

New patch with Gui update and variable renaming.

[StephaneP]

Circle from hdop value

[StephaneP]

Circle from horizontal deviation value

[StephaneP]

Circle from correction age value