[gdal-dev] Parllelization slows down single gdal_calc process in python

[Kor de Jong]

Dear Lorenzo,

On 03/03/2016 12:13 PM, Lorenzo Bottaccioli wrote:
> Yes I have 8 cores! The I/O output files are different for each process.
> I have to preform the gdal_calc on different maps each process. I just
> wanted to lunch more than one gdal_calc.py script at time.

Yes, I assumed the files used by each of your processes are different. 
This doesn't matter. They all have to be read and written by the one 
controller that provides access to the disk. It will serialize all 
access to the disk. You cannot read and/or write from/to multiple files 
that are located on the same disk in parallel.

You may have improved the performance of the computational part, but you 
also may have decreased the performance of the I/O part. The overall 
performance will depend on the balance between these two. In your case 
you may be better off optimizing for I/O. Instrumenting your code will 
provide you with clues about what is really going on.

Kor