[gdal-dev] gdal_rasterize in parallel processing
Giuseppe Amatulli
giuseppe.amatulli at gmail.com
Mon Sep 24 15:53:55 PDT 2012
Hi,
i'm running gdal_rasterize for several *.shp using xargs, sending
the process to 10 cpu.
1) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp |
xargs -n 1 -P 10 bash
/weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh
the same process can be run using one processor.
2) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp |
xargs -n 1 -P 1 bash
/weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh
Inputs and outputs are independent in each process so no issues in
reading/writing the same file.
The obtained tifs produced by the gdal_rasterize
(gdal_rasterize -a_nodata 0 -ot Float32 -co 'COMPRESS=LZW' -te
$geo_string_tif -tr 30 30 -l $filename -a BLOCKID10
$INDIR/$filename.shp $OUTDIR/$filename"_"$tile".tif" ;
where BLOCKID10 is an integer number if e.g 550 019 505 012 101 )
should be same but is not the case.
Line 2) produce the tif perfect similar with the attribute of the shp;
so no issues.
Line 1) burn the same BLOCKID10 value for several polygons, resulting
in less final polygons.
Seems to me that this phenomena manly effect BLOCKID10
values having several "0" (e.g. 10979900000033 ; 10030101002035)
May question is. Can be possible that the gdal driver used inside C
code not support multi-core processing?
If yes it is possible to find a solution to run gdal_rasterize in parallel.
Any idea?
Thanks in advance
Giuseppe
--
Web: www.spatial-ecology.net
More information about the gdal-dev
mailing list