[GRASS-user] RE: [GRASSLIST:1174] Working with very large dat
Brandon M. Gabler
bgabler at email.arizona.edu
Thu Aug 24 14:01:54 EDT 2006
Speaking of large datasets, I have an extremely large number of ArcInfo LIDAR
DEM tiles that I want to import into grass and subsequently join together; is
there a way to batch import files that are all different naming conventions? I
imagine there is a script somewhere that provides a GUI that allows more than
one file name input. These files total about 17GB worth of data, and
separate tiles means that they are unmanageable. But importing one at a
a waste of my time if there is a better way out there (especially if
there is a
way to import and patch at once?).
Brandon M. Gabler
Department of Anthropology
1009 E South Campus Drive, Building #30A
University of Arizona
Tucson, AZ 85721
Quoting "Patton, Eric" <epatton at nrcan.gc.ca>:
> Just to chime in on Hamish' r.in.xyz - I just got back from a field survey
> of the Bay of Fundy where 56GB of swath sonar (Simrad EM1002) was collected.
> After exporting each survey day from Caris HIPS as xyz, I've used r.in.xyz +
> r.patch to import the entire 56GB into Grass with no problems. I've always
> felt that r.in.xyz runs very quickly given the size of each xyz dataset.
> ~ Eric.
> -----Original Message-----
> From: grassuser-bounces at grass.itc.it
> To: Jonathan Greenberg; David Finlayson
> Cc: grassuser at grass.itc.it; Helena Mitasova; Andrew at grass.itc.it; Danner
> Sent: 8/24/2006 2:50 AM
> Subject: Re: [GRASS-user] RE: [GRASSLIST:1174] Working with very large
> David Finlayson wrote:
>> I am working with an interferometric sidescan SONAR system that
>> produces about 2 Gb of elevation and amplitude data per hour. Our raw
>> data density could support resolutions up to 0.1 m, but we currently
>> can't handle the data volume at that resolution so we decimate down to
>> 1 m via a variety of filters. Still, even at 1 m resolution, our
>> datasets run into the hundreds of Mb and most current software just
>> doesn't handle the data volumes well.
>> Any thoughts on processing and working with these data volumes (LIDAR
>> folks)? I have struggled to provide a good product to our researchers
>> using both proprietary (Fledermaus, ArcGIS) and non-proprietary (GMT,
>> GRASS, my own scripts) post-processing software. Nothing is working
>> very well. The proprietary stuff seems easier at first, but becomes
>> difficult to automate. The non-proprietary stuff is easy to automate,
>> but often can't handle the data volumes without first down sampling
>> the data density (GMT does pretty well if you stick to line-by-line
>> processing, but that doesn't always work).
>> Just curious what work flows/software others are using. In particular,
>> I'd love to keep the whole process FOSS if possible. I don't trust
>> black boxes.
> grassuser mailing list
> grassuser at grass.itc.it
More information about the grass-user