[Gdal-dev] running gdal_translate, gdaladdo, and gdaltindex simultaneously

Frank Warmerdam warmerdam at pobox.com
Mon Jan 15 13:26:35 EST 2007

John Mitchell wrote:
> Hi,
> I have set up 4 .bat files each which run the following over and over again:
> file1.bat                        file2.bat                          
> file3.bat                           file4.bat
> gdal_translate               gdal_translate                
> gdal_translate                  gdal_translate
> gdaladdo                      gdaladdo                       
> gdaladdo                          gdaladdo
> gdaltindex                    gdaltindex                      
> gdaltindex                        gdaltindex
> What I would like to do is run all 4 .bat files at the same time so as 
> to save time, but I am concerned that their may be a conflict since all 
> 4 files will be running gdaltindex to the same .shp file.  What if say 
> file1.bat and file3.bat happen to call gdaltindex at the same time will 
> they both fail to update the .shp file; will only one update the .shp or 
> will both of them successfully update the .shp file.  Is their any 
> parameter that I should add when running gdal_translate, gdaladdo or 
> gdaltindex to make it more efficient or stable?


This is indeed unsafe and could result in corruption of the index.  I
would suggest doing the gdal_translate, and gdaladdo "in parallel", but
doing the index creation in a single run afterwards. Index creation is
very fast compared to translation and adding overviews.

Note, if you are seeing surprisingly slow performance in gdal_translate
or gdaladdo it may be suffering from "cache thrashing", in which case
setting the GDAL_CACHEMAX environment variable to a reasonable size may


(for 200MB).

Best regards,
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org

More information about the Gdal-dev mailing list