[gdal-dev] gdal_rasterize in parallel processing

Giuseppe Amatulli giuseppe.amatulli at gmail.com
Mon Sep 24 15:53:55 PDT 2012


Hi,
i'm running  gdal_rasterize for  several *.shp using xargs, sending
the process to 10 cpu.

1) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp   |
xargs -n 1 -P 10  bash
/weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh

the same process can be run using one processor.

2) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp   |
xargs -n 1 -P 1  bash
/weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh

Inputs and outputs are independent in each process so no issues in
reading/writing the same file.

The obtained tifs  produced by the gdal_rasterize
(gdal_rasterize -a_nodata 0 -ot Float32 -co 'COMPRESS=LZW'  -te
$geo_string_tif   -tr 30 30    -l $filename     -a BLOCKID10
$INDIR/$filename.shp   $OUTDIR/$filename"_"$tile".tif"  ;
 where BLOCKID10 is an integer number if  e.g 550 019 505 012 101 )

should be same but is not the case.
Line 2) produce the tif perfect similar with the attribute of the shp;
so no issues.
Line 1) burn the same BLOCKID10 value for several polygons, resulting
in less final polygons.
           Seems to me that this phenomena manly effect BLOCKID10
values having several "0" (e.g. 10979900000033 ; 10030101002035)

May question is. Can be possible that the  gdal driver used inside C
code not support multi-core processing?
If yes it is possible to find a solution to run gdal_rasterize in parallel.
Any idea?

Thanks in advance
Giuseppe

-- 
Web: www.spatial-ecology.net


More information about the gdal-dev mailing list