[gdal-dev] gdal_rasterize in parallel processing

[Giuseppe Amatulli]

Hi all,
I solved the down reported issues. I change -ot Float32  to -ot
Float64 and it works.
Anyway Float32 should be enough to store the max value of
560459513003129 < 3.4E38
but probably during the multi-core several values are stored in the
same array and it needs Float64.


ciao ciao
Giuseppe

On 24 September 2012 17:53, Giuseppe Amatulli
<giuseppe.amatulli at gmail.com> wrote:
> Hi,
> i'm running  gdal_rasterize for  several *.shp using xargs, sending
> the process to 10 cpu.
>
> 1) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp   |
> xargs -n 1 -P 10  bash
> /weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh
>
> the same process can be run using one processor.
>
> 2) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp   |
> xargs -n 1 -P 1  bash
> /weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh
>
> Inputs and outputs are independent in each process so no issues in
> reading/writing the same file.
>
> The obtained tifs  produced by the gdal_rasterize
> (gdal_rasterize -a_nodata 0 -ot Float32 -co 'COMPRESS=LZW'  -te
> $geo_string_tif   -tr 30 30    -l $filename     -a BLOCKID10
> $INDIR/$filename.shp   $OUTDIR/$filename"_"$tile".tif"  ;
>  where BLOCKID10 is an integer number if  e.g 550 019 505 012 101 )
>
> should be same but is not the case.
> Line 2) produce the tif perfect similar with the attribute of the shp;
> so no issues.
> Line 1) burn the same BLOCKID10 value for several polygons, resulting
> in less final polygons.
>            Seems to me that this phenomena manly effect BLOCKID10
> values having several "0" (e.g. 10979900000033 ; 10030101002035)
>
> May question is. Can be possible that the  gdal driver used inside C
> code not support multi-core processing?
> If yes it is possible to find a solution to run gdal_rasterize in parallel.
> Any idea?
>
> Thanks in advance
> Giuseppe
>
> --
> Web: www.spatial-ecology.net



-- 
Giuseppe Amatulli
Web: www.spatial-ecology.net