[GRASS-dev] [SoC] Parallelization of Raster and Vector modules

[Jordan Neumeyer]

On Mon, Apr 12, 2010 at 11:35 AM, <Doug_Newcomb at fws.gov> wrote:

> >So you "tried" to do that? Does that mean you failed or is the system
> still processing it? :-P I kid, but that is a very large file... it's bigger
> than my current desktop's hard drive.
>
> Jordan,
>
> It would be a shame to leave those multicore  64 bit processors with
> multi-terabyte hard drives with nothing to challange them.  Just think of me
> as part of the hardware entertainment committee. :-)
>
Good, they need to be kept busy. Don't want to bore them. ;-)

> The process ran overnight and finished, I just have not gotten to phase 2
> of that particular project, chopping the resulting image into overlapping
> DOQQ - size images.  I was going to go back and change the projection method
> slightly, hopefully to make it more accurate.  If you're looking for big
> images to play with, go to http://datagateway.nrcs.usda.gov/  and select a
> naip county mosaic of imagery.  The mosaic comes as a MrSid format file, but
> once you uncompress it ( I suggest using gdal's nearblack utility ) to ERDAS
> Imagine format, the resulting image will probably be in the multi-gigbyte
> range per county. <http://datagateway.nrcs.usda.gov/>
>
Well that's good. It only took overnight for 500 GB? That doesn't sound too
bad. I'm not looking for huge maps, but some decent-sized samples would be
good for testing. Thanks for the resource I'll look into it if I need some
samples for testing. The warning at the bottom the site is a bit nerving
though... I'm not a government official.

>  Doug
>
>
>
>
> Doug Newcomb
> USFWS
> Raleigh, NC
> 919-856-4520 ext. 14 doug_newcomb at fws.gov
>
> ---------------------------------------------------------------------------------------------------------
> The opinions I express are my own and are not representative of the
> official policy of the U.S.Fish and Wildlife Service or Dept. of the
> Interior.   Life is too short for undocumented, proprietary data formats.
>
>
>  *Jordan Neumeyer <jordan.neumeyer at mines.sdsmt.edu>*
> Sent by: grass-dev-bounces at lists.osgeo.org
>
> 04/08/2010 01:09 AM
>   To
> Doug_Newcomb at fws.gov
> cc
> GRASS developers list <grass-dev at lists.osgeo.org>
> Subject
> Re: [GRASS-dev] [SoC] Parallelization of Raster and Vector modules
>
>
>
>
> Thanks guys for putting that into perspective.
>
> On Mon, Apr 5, 2010 at 6:38 PM, <*Doug_Newcomb at fws.gov*<Doug_Newcomb at fws.gov>>
> wrote:
>
> Doug
>
> Doug Newcomb
> USFWS
> Raleigh, NC
> 919-856-4520 ext. 14 *doug_newcomb at fws.gov* <doug_newcomb at fws.gov>
>
> ---------------------------------------------------------------------------------------------------------
> The opinions I express are my own and are not representative of the
> official policy of the U.S.Fish and Wildlife Service or Dept. of the
> Interior.   Life is too short for undocumented, proprietary data formats.
>
> -----*grass-dev-bounces at lists.osgeo.org*<grass-dev-bounces at lists.osgeo.org>wrote: -----
>
> To: Jordan Neumeyer <*jordan.neumeyer at mines.sdsmt.edu*<jordan.neumeyer at mines.sdsmt.edu>
> >
> From: Markus Neteler <*neteler at osgeo.org* <neteler at osgeo.org>>
> Sent by: *grass-dev-bounces at lists.osgeo.org*<grass-dev-bounces at lists.osgeo.org>
> Date: 04/05/2010 04:06AM
> cc: GRASS developers list <*grass-dev at lists.osgeo.org*<grass-dev at lists.osgeo.org>
> >
> Subject: Re: [GRASS-dev] [SoC] Parallelization of Raster and Vector modules
>
>
> On Sun, Apr 4, 2010 at 3:22 AM, Jordan Neumeyer
> <*jordan.neumeyer at mines.sdsmt.edu* <jordan.neumeyer at mines.sdsmt.edu>>
> wrote:
> > I didn't realize how big the data set could be. What's
> > biggest map you've seen?
>
> Our provincial DEM is a 3.5GB Geotiff which is of 48800x58000 size.
> Another file which I recently had to import was a 4GB Geotiff with
> 21550 bands. Finally, in remote sensing, you can quickly generate
> quickly files in the multi-GB range.
>
> Markus
> _______________________________________________
> grass-dev mailing list*
> **grass-dev at lists.osgeo.org* <grass-dev at lists.osgeo.org>*
> **http://lists.osgeo.org/mailman/listinfo/grass-dev*<http://lists.osgeo.org/mailman/listinfo/grass-dev>
>
> ~Jordan
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/grass-dev/attachments/20100415/c1f46159/attachment.html