[GRASS-user] Re: v.in-out quirks

[Shane Litherland]

(my first-ever reply to the list...hope I get the cc / subject stuff
right!)
(also FYI - GRASS 6.4.0RC, Ubuntu 10.04LTS (most of my sys/progs from
synaptic package manager)
AND a GARMIN GPS60)

Micha - your suggestion much appreciated :-) I had used v.in.ascii a
while back, had forgotten how versatile it could be!

I used a slight adjustment to your suggestion - I still like having
another copy of my gps data, so did it in the two steps rather than
piping, i.e. gpsbabel (gebabbel GUI) to save a unicsv format file from
the garmin handset, then the v.in.ascii (GRASS GUI) to get the unicsv
format into grass.
Having the csv file also meant I could 'clean' it up (remove a few ""
from around text/numbers) so it went into a Postgres table much neater
when i ran v.in.ascii
I played with a few other format options in gpsbabel; garmin_txt and
openoffice formats 'found' all the data though the txt files probably
needed cleaning up (excess empty fields) before taking into
GRASS/Postgres.

In a nutshell, it seems my 'lost info problem' may lie with my GPS and
GPSBABEL - my model records the date/time for a waypoint as a
'note' (e.g. 06-DEC-10 20:15:40), hence gpsbabel finds it as
text/description. Looking at some of the source styles and other
C-source-code files of gpsbabel, it seems this type of date/time/note
doesn't match what gpsbabel would expect as a proper date/time. Thought
I'd try and streamline my tasks with my own style code, got the waypoint
'names' recognised, but hit a wall with the 'strptime'-based date/time
formatting options... something to take up on a gpsbabel forum..

So for now I stick with the workaround :-)

Also -
Using Postgres as db, I had the option of choosing a date-time field as
"timestamp without time zone" which is an option that didn't seem
available in the default dbf... so it can convert the date/time string
to a proper date and time - which means I can get away with having
gpsbabel output the date/time as one text field... I like Postgres :-)

v.in.garmin:
I think the reason I could not get v.in.garmin to work, is that it uses
GPSTRANS (or GARDUMP) as the means to download the data from the gps in
the first steps. So the issue there, is more with gpstrans than
v.in.garmin. From all my reading and experimenting, it seems these
programs are geared at communicating with a serial or usb-to-serial gps
unit. My unit is 'proper' usb (i.e. no /dev/tyyUSB0 etc, 'lsusb' in
terminal shows it connects to /dev/bus/usb/) but despite my
experimenting, i could not get gpstrans to find the usb ports. Didn't
try gardump yet. The above workaround is easier than fiddling with more
unfamilar software ;-) From what I read, both these have been around a
while, and not so much has happened with them of late?? maybe
v.in.garmin could be rolled into v.in.gpsbabel, with gpstrans and
gardump as second or third options to gpsbabel... if it's not something
that's already been improved since my 6.4 version!

v.in.gpsbabel:
If I edited a csv file, to change field names, edit date/time to correct
format etc, and ran this into v.in.gpsbabel, I got all the right info
out the other side (i.e. vector points and data in db table). So,
similar to above, it seems that the fields/format that gpsbabel attaches
to my gps data, don't get recognised accurately when being used by
v.in.gpsbabel. i.e. the issue is not really with GRASS.

But If anyone is interested in following this gps-connection stuff, to
improve v.in.garmin or v.in.gpsbabel, I have had success communicating
with my usb-gps in other programs (e.g. Viking, Prune) so there may be
ideas to get from there... Short of taking 'garmin_gps' off the modprobe
'blacklist', I have NOT had any luck getting my gps to connect to
gpstrans, gpsman or QlandkarteGT 0.17.1 from ubuntu rep's (yet to try
latest QlandkarteGT which apparently had work done on garmin issues). So
no tips to offer from there :-/

v.in.ogr:
- done a bit more experimenting... still doing more... there might be
something to check in regards to v.in.ogr using 'desc' as a field name
and causing the database (Postgres in my case) to be upset by a reserved
word being used as a field/column name.. if I get a chance to look at
the source code (have figured out I can do that from source downloads,
yay!) I'll see if I can narrow this one down too..

Hope someone finds a bit of useful stuff in all that rambling!

regards,
shane.

On Fri, 2010-12-03 at 14:07 +0200, Micha Silver wrote:
> On 12/02/2010 02:01 PM, Shane Litherland wrote:
> > hi all you GRASSers ;-)
> > not sure what happened when i first tried to join the mailing list a few
> > months ago, but i got my user/subscription sorted now :-)
> >
> > I have been using GRASS for a couple years now, more consistently in the
> > last 12 months after a steady learning process. So hopefully as I come
> > across 'things' there's less likelihood it's just me being a silly
> > novice (less, but not a surety..)
> >
> > my latest fiddling has been with import of GPS data by a few different
> > means, and some issues I struck.
> >
> > my methods:
> > GPS to .gpx file using GEBABBEL (GUI for gpsbabel);
> > .gpx file has:
> > wpt (waypoints co-ords lat-long)
> > ele (elevation)
> > name (the name/number I gave each waypoint when I marked them)
> > cmt (supposedly a comment, it is the date/time)
> > desc (date/time again)
> > sym (name of symbol used in gps)
> >    
> 
> Here's an idea that might work for you (tested only briefly...)
> gpsbabel has a file format 'unicsv' that throws the data into a comma 
> separated list, with headers. That can be piped straight into v.in.ascii 
> with some careful choice of parameters.
> Here's what worked for me:
> 
> gpsbabel -t -i gpx -o unicsv -f bike_rides_1208.gpx -F - | v.in.ascii 
> out=bike_rides2 fs=, col="num integer, y_coord double, x_coord double, 
> elev double, date_str varchar(16), time_str varchar(16)" x=3 y=2 skip=1
> 
> I had a gpx file already downloaded from the gps, but no reason this 
> wouldn't work pulling the data right off the unit. The gpsbabel command 
> include "-F -" (note the trailing dash) to indicate that the output just 
> goes to STDOUT. Then it's piped into v.in.ascii (inside a grass session, 
> of course). You need the "fs=," since the csv is indeed comma separated. 
> Then you use the col="..." to give any name/type you want to each of the 
> gps attributes. Next be careful to choose the correct column numbers for 
> X and Y (duh!) and finally add "skip=1" to skip the first line of gps 
> column headers.
> 
> If this helps, please post back...
> -- 
> Micha
> 
> > v.in.ogr to import gpx file to GRASS;
> > using Postgres as database;
> > specifying just one layer (e.g. waypoints);
> > ... get an error when v.in.ogr tries to create table in postgres -
> > doesn't like 'desc' as a column name (one of those special db words).
> > ...OK, so I can change the column names with cnames, but then it was
> > upset by 'time datetime' that corresponded to what should be 'name
> > varchar' in the .gpx file.
> > ... or change 'desc' to 'description' in the .gpx file- but no, the
> > error still happens, seems the 'desc' cname is built into the v.in.ogr,
> > it's not checking any of the cnames with the input file
> > ...but I also note in the output that the columns generated by the
> > v.in.ogr are way more than I have in the .gpx file, and the column
> > formats for the first few columns don't all match the formats for the
> > data they'd 'find' in the .gpx file.
> > ... was wondering how then, I might get to the part of the v.in.ogr
> > module(?) that has all these columns to edit it...?? I did try in the
> > cnames, including 'name format' like i'd do when making a table, but
> > that confused the poor v.in.ogr (showed 'name format format' in the
> > output and didn't like making a table that way!)
> > ... can check the box to 'not create attribute table' and import it OK
> > but of course, any waypoint name/info doesn't make it into GRASS /
> > postgres.
> > ...OK, so v.in.ogr might not be the way to get .gpx stuff into grass
> > anyway, but i'm trying things out here!
> >
> > my next approach-
> > v.in.gpsbabel;
> > using .gpx format;
> > ... it runs OK, but generates a table with:
> > cat (added for the db)
> > x (lat)
> > y (long)
> > altitude (elevation)
> > gmt_date (blank - zero string length when table created)
> > gmt_time (blank - ditto)
> > comments (obvious)
> >
> > when what was in the .gpx file was really
> >
> > (cat - to be added for the db)
> > x
> > y
> > ele
> > name
> > cmt ('comment' actually datetime)
> > description (also datetime)
> > sym (name of symbol used in gps)
> >
> > ... so, whilst the v.in.gpsbabel got the waypoints in, it too 'lost' the
> > name of each waypoint and mucked up the date/time field/format when
> > populating the table.
> >
> > I have not tried creating a db table directly (i.e. in postgres) and
> > then creating vector in GRASS and connecting them, as I figure the only
> > field i could safely link them on would be the datetime, because cats
> > generated during vector import probably wouldn't match the cats I made
> > manually in the table... and if the datetime was mucked up in its format
> > (e.g. as varchar instead) ... well, I just reckon that option could be
> > rather messy ;-)
> >
> > Of course, I seem to have 'missed' the obvious option of directly
> > importing from the GPS unit to GRASS... well, I tried, but had to go and
> > figure out how the GPS connected to my computer (in Ubuntu 10.04, the
> > default connection shown in v.in.garmin of /dev/gps wasn't found)
> > ...after getting fresh batteries for my gps, I tried connecting via:
> > /dev/usb
> > /dev/usbmon0 (thru to mon7)
> > /dev/ttyS0
> > /dev/USB0
> > /dev/tty0
> > usb:
> > ...these 'ideas' from bits I saw in other GUI's and/or on the web, but
> > to no avail - the receiver did not respond on any of these ports.
> > I know gpsbabel (gebabbel GUI) connected to the unit with the "usb:"
> > setting because that's how I made my .gpx file in the first place.
> >
> > I know of other 'workarounds' e.g. I have (a while ago) run the old
> > Mapsource in Wine, and then get the waypoints/tracks etc into a format
> > for GRASS, had also played with gpsman and some other bits. But I'm not
> > elaborating on those workarounds, at the moment it's more about sharing
> > my experience on how the more direct options didn't-quite-work ;-)
> >
> > At the end of all that, I guess if I knew how to edit the v.in.ogr /
> > v.in.garmin / etc, I could tweak the bits to work..? but (I find this
> > bemusing still) these are .bin files, so despite them being part of an
> > open-source software, they are, from my perspective, still 'closed' as
> > far as being able to know what's going on in there ;-) and i'm not sure
> > i'm quite skilled enough to play too much with those binary-readers... i
> > could do something wrong and break things...
> >
> > That's my 'quirk of the month' session done :-) I hear the sighs of
> > relief!
> >
> > As an aside, I've been considering putting in bits for manuals/tutorials
> > along the way - my use of GRASS is more-or-less from an ecological and
> > land-management perspective and not in-depth, but OK for e.g.
> > entry/undergraduate level users... thoughts/contacts welcome on this one
> > too.
> >
> > Regards,
> > Shane.
> >
> > _______________________________________________
> > grass-user mailing list
> > grass-user at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/grass-user
> >
> > This mail was received via Mail-SeCure System.
> >
> >
> >    
> 
>