[Tilecache] Re: Concurrent Seeding

[bwoodall]

Howdy Bill,

Just a 1st guess, but it sounds like an iowait issue ( getting the data
on & off the drives ).  Have you already looked at this? if not, then a
quick ( though maybe not the best ) way is to see what the 'top' command
tells you, look ( mine is the 3rd line down) for the "Cpu(s)" line  and
then look for "%wa", which is the iowait.

>From the top man page; "%wa - Amount of time the CPU has been waiting
for I/O to complete.".

$,02

..Bill,

On Thu, 2011-06-16 at 22:31 -0700, Bill Teluk wrote:
> Update:
> I've written a Bash script based on Tedman Eng's multiple process script as
> published in the "Advanced Bash Scripting Guide" (Mendel Cooper) and I can
> get multiple instances of the the seeding program to run, each with an
> individual bbox and zoom range.  The script can be reconfigured to vary the
> number of processes to execute concurrently.
> However, I don't get any signficant improvement when running anything more
> than about 3 concurrent.  My CPU is about 83% idle!!! (that's on a 24 CPU
> box with 48G of RAM.)  And the resulting speed is about 2.4 times the amount
> of time to generate tiles in serial.  In fact, increasing the number of
> concurrent processes, eg. to 10 or 20, actually slows down the tile
> generation rate slightly.
> 
> Is there some sort of common resource that TileCache uses that get's locked
> or otherwise acts as a bottleneck when seeding?  Or is there some method by
> which I can tweak the system to get it to work faster?
> 
> I'm using modPython, an FGS v9.5 MapServer install on a RHEL5.5 Linux,
> PostgreSQL v9.0.3, PostGIS 1.5.2, the layer is sourced from Postgres and is
> a street directory map based on some 35 odd layers (100 classes) in the Map
> file.
> 
> Regards, Bill Teluk
> 
> --
> View this message in context: http://osgeo-org.1803224.n2.nabble.com/Concurrent-Seeding-tp6444150p6485820.html
> Sent from the TileCache mailing list archive at Nabble.com.
> _______________________________________________
> Tilecache mailing list
> Tilecache at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/tilecache