[gdal-dev] gdal_rasterize in multiple processes

[Nicolas Cadieux]

Hi,

Are you launching multiple at the same time or are you looping and treating one file at a time?  If so, your computer is just waiting to the last file to finish before it starts the next.  When the file are smaller, they appear to be running at the same time but when the files get bigger, the computer needs to wait.  The hard drive speed or memory band width are typical bottle necks that can slow down a very fast machine to a trickle if they get saturated.


Nicolas Cadieux M.Sc.
Les Entreprises Archéotec inc. 
8548, rue Saint-Denis Montréal H2P 2H2
Téléphone: 514.381.5112  Fax: 514.381.4995
www.archeotec.ca

On Jul 14, 2015 11:25, jramm <jamessramm at gmail.com> wrote:
>
> I have a number of vector datasets to rasterize. 
>
> I am finding that when the datasets are small, I have no problem starting 
> multiple gdal_rasterize processes (using python). 
>
> If my datasets are large (I'm trying to create rasters with millions of 
> pixels), I'm finding that I can only start one gdal_rasterize process at 
> once, or sometimes, after a couple have started the rest refuse to start. 
>
> RAM shows usage below 50%, and CPU usage is also below 50%. 
>
> I can't figure out whether this is a gdal_rasterize thing or not? 
>
>
>
> -- 
> View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-rasterize-in-multiple-processes-tp5215591.html 
> Sent from the GDAL - Dev mailing list archive at Nabble.com. 
> _______________________________________________ 
> gdal-dev mailing list 
> gdal-dev at lists.osgeo.org 
> http://lists.osgeo.org/mailman/listinfo/gdal-dev