[GRASS-user] RE: [GRASSLIST:1174] Working with very large dat asets

[Patton, Eric]

No, actually all acquisition was done on Windows computers. Our navigation
software only runs on Windows as well. I'm going to try buying my own laptop
and using Ubuntu Linux with Grass 6.3 in the field next survey season to
test that out. We never seem to have an adequate GIS in the field during
data acquisition. I'd like to be able to test out a bunch of scripts that I
wrote for converting from our navigation trackline format to Grass vector
format. I think your v.swathwidth would also come in handy in survey
planning as well. 

I've never tried compiling Grass on Cygwin so I'm not much help there.  

~ Eric.

-----Original Message-----
From: David Finlayson
To: Patton, Eric
Cc: Hamish; Jonathan Greenberg; grassuser at grass.itc.it ; Helena Mitasova;
Andrew at grass.itc.it ; Danner
Sent: 8/24/2006 4:19 PM
Subject: Re: [GRASS-user] RE: [GRASSLIST:1174] Working with very large dat
asets

I assume that you were in a linux environment on your ship's network?

Our aquisition and navigation software run only on Windows machines.
Sidescan is processed on Windows, RTK GPS is on Windows, I have some
flexibility on bathy processing but you can see the pattern... 

I just updated the cygwin version of grass 6.1.cvs and it doesn't have
r.in.xyz.
Cygwin grass is kind-of a mixed bag anyway...

How much work is involved compiling Grass on cygwin? I've only compiled
it on Linux. 



On 8/24/06, Patton, Eric <epatton at nrcan.gc.ca
<mailto:epatton at nrcan.gc.ca> > wrote:

Just to chime in on Hamish' r.in.xyz - I just got back from a field
survey
of the Bay of Fundy where 56GB of swath sonar (Simrad EM1002) was
collected.
After exporting each survey day from Caris HIPS as xyz, I've used
r.in.xyz +
r.patch to import the entire 56GB into Grass with no problems. I've
always
felt that r.in.xyz runs very quickly given the size of each xyz dataset.

~ Eric.

-----Original Message-----
From: grassuser-bounces at grass.itc.it
<mailto:grassuser-bounces at grass.itc.it> 
To: Jonathan Greenberg; David Finlayson
Cc: grassuser at grass.itc.it <mailto:grassuser at grass.itc.it> ; Helena
Mitasova;  <mailto:Andrew at grass.itc.it> Andrew at grass.itc.it; Danner
Sent: 8/24/2006 2:50 AM
Subject: Re: [GRASS-user] RE: [GRASSLIST:1174] Working with very large
datasets

David Finlayson wrote:
> I am working with an interferometric sidescan SONAR system that 
> produces about 2 Gb of elevation and amplitude data per hour. Our raw
> data density could support resolutions up to 0.1 m, but we currently
> can't handle the data volume at that resolution so we decimate down to

> 1 m via a variety of filters. Still, even at 1 m resolution, our
> datasets run into the hundreds of Mb and most current software just
> doesn't handle the data volumes well.
>
> Any thoughts on processing and working with these data volumes (LIDAR 
> folks)? I have struggled to provide a good product to our researchers
> using both proprietary (Fledermaus, ArcGIS) and non-proprietary (GMT,
> GRASS, my own scripts) post-processing software. Nothing is working 
> very well. The proprietary stuff seems easier at first, but becomes
> difficult to automate. The non-proprietary stuff is easy to automate,
> but often can't handle the data volumes without first down sampling 
> the data density (GMT does pretty well if you stick to line-by-line
> processing, but that doesn't always work).
>
> Just curious what work flows/software others are using. In particular,
> I'd love to keep the whole process FOSS if possible. I don't trust 
> black boxes.






-- 
David Finlayson