[GIS] Convert NMEA to GPX in GPSBabel

convertgpsbabelgpxnmea

I have received NMEA data stored in an excel file. Sample data:

DOS_TXT GPS_WS  NMEA    D
1530000114  379550,0702 $GPGGA,092550.000,6311.7019,N,01429.1063,E,1,09,0.88,357.2,M,33.6,M,,*6B,5567,5567736,  TRM_NMEA
1530000115  379550,0809 $GPGSA,A,3,08,21,11,10,13,32,15,18,27,,,,1.18,0.88,0.77*0B,5581,5582816,    TRM_NMEA
1530000116  379550,0935 $GPRMC,092550.000,A,6311.7019,N,01429.1063,E,0.01,215.98,271016,,,A*60,5593,5593216,    TRM_NMEA
1530000117  379550,0999 $GPVTG,215.98,T,,M,0.01,N,0.02,K,A*39,5605,5606104, TRM_NMEA

The GPSBabel code looks like that in the GUI:

gpsbabel -w -t -i nmea -f /Users/.../GPS_DATA_20161027.xlsx -o gpx -F  /Users/.../testflight.gpx

I just get an empty file and don't know what's the issue.

Best Answer

I extend my comment to an answer. If you set input file format to NMEA, then you must provide a valid NMEA file. An XLS won't be accepted. So erase all the columns from your xls except NMEA column. Export it into a text file (csv or txt). There are some extra data at the end of the NMEA data. The end of NMEA data is a checksum stating with asterix (*), for example *6B for the first line. I erased data after checksum. Your file should look like this (I put it into test.nmea):

$GPGGA,092550.000,6311.7019,N,01429.1063,E,1,09,0.88,357.2,M,33.6,M,,*6B
$GPGSA,A,3,08,21,11,10,13,32,15,18,27,,,,1.18,0.88,0.77*0B
$GPRMC,092550.000,A,6311.7019,N,01429.1063,E,0.01,215.98,271016,,,A*60
$GPVTG,215.98,T,,M,0.01,N,0.02,K,A*39

Using the following command:

gpsbabel -w -t -i nmea -f test.nmea -o gpx -F test.gpx

I got this in test.gpx:

<?xml version="1.0" encoding="UTF-8"?>
<gpx version="1.0" creator="GPSBabel - http://www.gpsbabel.org" xmlns="http://www.topografix.com/GPX/1/0">
  <time>2017-01-12T16:40:59.396Z</time>
  <bounds minlat="63.195031667" minlon="14.485105000" maxlat="63.195031667" maxlon="14.485105000"/>
  <trk>
    <trkseg>
      <trkpt lat="63.195031667" lon="14.485105000">
        <ele>357.200000</ele>
        <time>2016-10-27T09:25:50Z</time>
        <course>215.979996</course>
        <speed>0.005556</speed>
        <geoidheight>33.6</geoidheight>
        <fix>3d</fix>
        <sat>9</sat>
        <hdop>0.880000</hdop>
        <vdop>0.770000</vdop>
        <pdop>1.180000</pdop>
      </trkpt>
    </trkseg>
  </trk>
</gpx>

I used gpsbabel Version 1.5.2. on Ubuntu 16.04. Comparing your data to the NMEA standard (http://www.gpsinformation.org/dale/nmea.htm), some sentences look invalid.

Related Solutions

Building a Geographic Database from GPS Logs – How to Guide

ptrv/gpx2spatialite does this remarkably well, saving timestamps for all points and deriving speed and length data for tracks. It also won't import duplicate tracks, so you can feed it a huge pile for GPX files and it will munge them appropriately.

Update: usage examples, as requested:

Initialize new database:

gpx2spatialite_create_db db.sqlite

Add a single gpx file:

gpx2spatialite -d db.sqlite -u user file.gpx

(If username user does not exist in the database, will query if you wish to create it. The database is built with tracks/waypoints/routes assigned to one or more users.)

Add all of the gpx files in folder/:

gpx2spatialite -d db.sqlite -u user folder/

Usage: it's a spatial database, with table tracklines holding the tracks as polylines, trackpoints containing the individual points (with time, elevation, etc), and waypoints for any defined waypoints. There are also numerous indices, some world city definitions and possibly also embedded SLD styling information I haven't yet explored. As a trivial example, here's a map of how my hometown (Toronto) is defined through ~7 years of track logs:

Toronto GPS logs

By using lines at 75% transparency, the most frequently-travelled routes become darker.

[GIS] NMEA to CSV with timestamp using gpsbabel

Thanks for your update with some sample data. As one can see from your sample, the nmea format basically already is csv but values might occur in rather uncommon formats and different nmea sentences (the one starting with $) often store the same information simply in a different format until the next new piece of information occurs. Hence, it is most often enough to evaluate all lines of only one of the sentences that holds the information that you are interested in.

Since I don't know what exact information you need (some of the sentences store really complicated stuff regarding the GPS signal), in the following I went for values for date, time, lat, lon, speed. However, it should not be too difficult to adapt the example to something else, otherwise just ask.

See inline comments for further explanation. In order to run this script, save the code in a .py file, adapt the INPUT_FILENAME and OUTPUT_FILENAME variables at the top of it to your needs, then just run Python filename.py in a console.

import csv
from datetime import datetime
import math

# adapt this to your file
INPUT_FILENAME = 'in.nmea'
OUTPUT_FILENAME = 'out.csv'

# open the input file in read mode
with open(INPUT_FILENAME, 'r') as input_file:

    # open the output file in write mode
    with open(OUTPUT_FILENAME, 'wt') as output_file:

        # create a csv reader object from the input file (nmea files are basically csv)
        reader = csv.reader(input_file)

        # create a csv writer object for the output file
        writer = csv.writer(output_file, delimiter=',', lineterminator='\n')

        # write the header line to the csv file
        writer.writerow(['date_and_time', 'lat', 'lon', 'speed'])

        # iterate over all the rows in the nmea file
        for row in reader:

            # skip all lines that do not start with $GPRMC
            if not row[0].startswith('$GPRMC'):
                continue

            else:

                # for each row, fetch the values from the row's columns
                # columns that are not used contain technical GPS stuff that you are probably not interested in
                time = row[1]
                warning = row[2]
                lat = row[3]
                lat_direction = row[4]
                lon = row[5]
                lon_direction = row[6]
                speed = row[7]
                date =  row[9]

                # if the "warning" value is "V" (void), you may want to skip it since this is an indicator for an incomplete data row)
                if warning == 'V':
                    continue

                # merge the time and date columns into one Python datetime object (usually more convenient than having both separately)
                date_and_time = datetime.strptime(date + ' ' + time, '%d%m%y %H%M%S.%f')

                # convert the Python datetime into your preferred string format, see http://www.tutorialspoint.com/python/time_strftime.htm for futher possibilities
                date_and_time = date_and_time.strftime('%y-%m-%d %H:%M:%S.%f')[:-3] # [:-3] cuts off the last three characters (trailing zeros from the fractional seconds)

                # lat and lon values in the $GPRMC nmea sentences come in an rather uncommon format. for convenience, convert them into the commonly used decimal degree format which most applications can read.
                # the "high level" formula for conversion is: DDMM.MMMMM => DD + (YY.ZZZZ / 60), multiplicated with (-1) if direction is either South or West
                # the following reflects this formula in mathematical terms.
                # lat and lon have to be converted from string to float in order to do calculations with them.
                # you probably want the values rounded to 6 digits after the point for better readability.
                lat = round(math.floor(float(lat) / 100) + (float(lat) % 100) / 60, 6)
                if lat_direction == 'S':
                    lat = lat * -1

                lon = round(math.floor(float(lon) / 100) + (float(lon) % 100) / 60, 6)
                if lon_direction == 'W':
                    lon = lon * -1

                # speed is given in knots, you'll probably rather want it in km/h and rounded to full integer values.
                # speed has to be converted from string to float first in order to do calculations with it.
                # conversion to int is to get rid of the tailing ".0".
                speed = int(round(float(speed) * 1.852, 0))

                # write the calculated/formatted values of the row that we just read into the csv file
                writer.writerow([date_and_time, lat, lon, speed])

Running this code on your sample, it will produce a csv file with the content of

date_and_time,lat,lon,speed
16-02-04 09:20:27.696,14.649645,121.072897,2
16-02-04 09:20:28.696,14.649658,121.07297,3

Note that this is all there is, all other lines only store redundant information with regards to the columns that are analyzed.

Best Answer

Related Solutions

Building a Geographic Database from GPS Logs – How to Guide

[GIS] NMEA to CSV with timestamp using gpsbabel

Related Question