[gdal-dev] Re: Strange things with gdalwarp ...

[Hermann Peifer]

Frank Warmerdam wrote
> Jukka Rahkonen wrote:
>>
>> Conclusion: Too slow to be useful with my settings.
>> Question 1: Should it work this way?
>> Question 2: If yes, is it worth trying to find the limits when it goes 
>> too slow
>> and perhaps file a ticket?
> 
> 
> I hate to bring this up so late in the discussion, but have you tried
> -wo SKIP_NOSOURCE=YES?  When mosaicing small images into large images it
> prevents processing of chunks for which there is no source data.  Otherwise
> the whole output image is generated for each input image.
> 
> There is an open ticket on this issue to apply this option by default.
> 
> BTW, in combination with SKIP_NOSOURCE it can be helpful to keep the -wm
> option to a modest size to the tiles aren't too large.  A 200MB should be
> sufficient - especially if the input and output image are tiled.
> 

Just to repeat my earlier observations: over the last days, I tried with all kinds of -wm and GDAL_CACHEMAX settings, and also added the options TILED=YES and SKIP_NOSOURCE=YES. I haven't documented the performance changes systematically, but my overall impression was that the changes were rather small. I also tried -multi on my dual processor machine, but I ended up with a fatal error.

What really helped was to follow Felix' approach, who wrote in the first mail of this thread:
> Now, a second script, pasting first "slices" of 5*30° together,
> and than merging all the slices...

This way, one can reduce the processing time to some 10-20% compared to the "all in one run approach". Concerning the size of the intermediate slices: the smaller they are, the faster it goes. The final step of gdalbuilding a vrt from the imtermediate slices and gdal_translating it to the final output.tif is also very fast.

In order to avoid NODATA gaps in the final output.tif: It seems to be safer if the intermediate slices are slightly overlapping, rather being than just adjacent.

Hermann