[gdal-dev] Re: Strange things with gdalwarp ...

[Adam Nowacki]

Some rather counterintuitive gdalwarp behavior: the bigger 
dfWarpMemoryLimit (-wm setting) the more cpu time will be wasted on 
warping not existing pixels. Why? Warping begins with the destination 
window size of entire output image size. If this size is larger than 
dfWarpMemoryLimit it is split in half along the longest axis and any 
half that doesn't contain the currently processed source file is 
discarded. With large dfWarpMemoryLimit this subdivision process will 
stop early with still large portions of out of source image pixels.

Hermann Peifer wrote:
> Frank Warmerdam wrote
>> Jukka Rahkonen wrote:
>>>
>>> Conclusion: Too slow to be useful with my settings.
>>> Question 1: Should it work this way?
>>> Question 2: If yes, is it worth trying to find the limits when it 
>>> goes too slow
>>> and perhaps file a ticket?
>>
>>
>> I hate to bring this up so late in the discussion, but have you tried
>> -wo SKIP_NOSOURCE=YES?  When mosaicing small images into large images it
>> prevents processing of chunks for which there is no source data.  
>> Otherwise
>> the whole output image is generated for each input image.
>>
>> There is an open ticket on this issue to apply this option by default.
>>
>> BTW, in combination with SKIP_NOSOURCE it can be helpful to keep the -wm
>> option to a modest size to the tiles aren't too large.  A 200MB should be
>> sufficient - especially if the input and output image are tiled.
>>
> 
> Just to repeat my earlier observations: over the last days, I tried with 
> all kinds of -wm and GDAL_CACHEMAX settings, and also added the options 
> TILED=YES and SKIP_NOSOURCE=YES. I haven't documented the performance 
> changes systematically, but my overall impression was that the changes 
> were rather small. I also tried -multi on my dual processor machine, but 
> I ended up with a fatal error.
> 
> What really helped was to follow Felix' approach, who wrote in the first 
> mail of this thread:
>> Now, a second script, pasting first "slices" of 5*30° together,
>> and than merging all the slices...
> 
> This way, one can reduce the processing time to some 10-20% compared to 
> the "all in one run approach". Concerning the size of the intermediate 
> slices: the smaller they are, the faster it goes. The final step of 
> gdalbuilding a vrt from the imtermediate slices and gdal_translating it 
> to the final output.tif is also very fast.
> 
> In order to avoid NODATA gaps in the final output.tif: It seems to be 
> safer if the intermediate slices are slightly overlapping, rather being 
> than just adjacent.
> 
> Hermann
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
>