[GRASS-user] RE: [GRASSLIST:1174] Working with very large dat asets

[Brandon M. Gabler]

Speaking of large datasets, I have an extremely large number of ArcInfo LIDAR
DEM tiles that I want to import into grass and subsequently join together; is
there a way to batch import files that are all different naming conventions? I
imagine there is a script somewhere that provides a GUI that allows more than
one file name input. These files total about 17GB worth of data, and 
leaving as
separate tiles means that they are unmanageable. But importing one at a 
time is
a waste of my time if there is a better way out there (especially if 
there is a
way to import and patch at once?).

Thanks,
Brandon


--
Brandon M. Gabler
Research Associate
Department of Anthropology
1009 E South Campus Drive, Building #30A
University of Arizona
Tucson, AZ 85721
Phone: 520-621-8455
Fax: 520-621-2088


Quoting "Patton, Eric" <epatton at nrcan.gc.ca>:

> Just to chime in on Hamish' r.in.xyz - I just got back from a field survey
> of the Bay of Fundy where 56GB of swath sonar (Simrad EM1002) was collected.
> After exporting each survey day from Caris HIPS as xyz, I've used r.in.xyz +
> r.patch to import the entire 56GB into Grass with no problems. I've always
> felt that r.in.xyz runs very quickly given the size of each xyz dataset.
>
> ~ Eric.
>
> -----Original Message-----
> From: grassuser-bounces at grass.itc.it
> To: Jonathan Greenberg; David Finlayson
> Cc: grassuser at grass.itc.it; Helena Mitasova; Andrew at grass.itc.it; Danner
> Sent: 8/24/2006 2:50 AM
> Subject: Re: [GRASS-user] RE: [GRASSLIST:1174] Working with very large
> datasets
>
> David Finlayson wrote:
>> I am working with an interferometric sidescan SONAR system that
>> produces about 2 Gb of elevation and amplitude data per hour. Our raw
>> data density could support resolutions up to 0.1 m, but we currently
>> can't handle the data volume at that resolution so we decimate down to
>> 1 m via a variety of filters. Still, even at 1 m resolution, our
>> datasets run into the hundreds of Mb and most current software just
>> doesn't handle the data volumes well.
>>
>> Any thoughts on processing and working with these data volumes (LIDAR
>> folks)? I have struggled to provide a good product to our researchers
>> using both proprietary (Fledermaus, ArcGIS) and non-proprietary (GMT,
>> GRASS, my own scripts) post-processing software. Nothing is working
>> very well. The proprietary stuff seems easier at first, but becomes
>> difficult to automate. The non-proprietary stuff is easy to automate,
>> but often can't handle the data volumes without first down sampling
>> the data density (GMT does pretty well if you stick to line-by-line
>> processing, but that doesn't always work).
>>
>> Just curious what work flows/software others are using. In particular,
>> I'd love to keep the whole process FOSS if possible. I don't trust
>> black boxes.
>
> _______________________________________________
> grassuser mailing list
> grassuser at grass.itc.it
> http://grass.itc.it/mailman/listinfo/grassuser