[GRASS-user] GRASS 7 Bottleneck

[Sören Gebbert]

Hi Seth,
more inline:

2012/2/28 Seth Price <seth at pricepages.org>:
> I'm running a few instances of GRASS, and I've hit an extreme bottleneck. The problem is that I can't figure out what it is.
>
> I have a module that I wrote in the GRASS 7.0 environment that should be I/O bound. It normally takes about 10 minutes to finish running. I have seven instances running from one disk, and each one is taking on the order of 10 hours to finish. Why?
>
> NOTE:
> - The processes aren't I/O bound. 'iostat' lists a constant 30 MB/s transfer rate. It should be able to handle twice that. The transfers per second are < 100, and I've seen it handle triple that (at 60 MB/s).
> - The load average is currently below 1. I have 16 processors, so CPU shouldn't be a problem.
> - I have 16 GB of RAM, and it's full, but not thrashing.

Since you have plenty of RAM, the Linux system might buffer a lot of
the module input/output in the main memory. So the bottleneck may
appear when Linux tries to write the buffered output to the disc.
Does the load value not change in the 10h of processing? Is the
transfer rate constant over the 10h?
My guess is that when your module reads and writes permanently to the
file system it is IO bound and your hard disk is the limiting factor
not the GRASS (raster, vector or voxel) IO implementation.

I assume the seven instances writing to different maps?

For comparison: I have a 24 Core + 64Gig Ram number cruncher, running
24 parallel processes of v.surf.idw for interpolation purpose. HD's
are 4x256GB SSD's in a fast raid configuration. Max write speed i have
measured is about 400MByte/s with large files. This scales very well
without any performance issues with 24 processes which are not IO
bound.

Best regards
Soeren

>
> If it's not I/O, CPU, or RAM, then what is it slowing GRASS?
> Thanks,
> Seth_______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user