[gdal-dev] Re: Strange things with gdalwarp ...

[Hermann Peifer]

Adam Nowacki wrote:
> Some rather counterintuitive gdalwarp behavior: the bigger 
> dfWarpMemoryLimit (-wm setting) the more cpu time will be wasted on 
> warping not existing pixels. Why? Warping begins with the destination 
> window size of entire output image size. If this size is larger than 
> dfWarpMemoryLimit it is split in half along the longest axis and any 
> half that doesn't contain the currently processed source file is 
> discarded. With large dfWarpMemoryLimit this subdivision process will 
> stop early with still large portions of out of source image pixels.
> 

In other words, the counter-intuitive recommendation would be: The smaller the -wm setting, the better?
Or say: the -wm value should better be just a bit bigger than the source image size?

Hermann



> Hermann Peifer wrote:
>> Frank Warmerdam wrote
>>> Jukka Rahkonen wrote:
>>>>
>>>> Conclusion: Too slow to be useful with my settings.
>>>> Question 1: Should it work this way?
>>>> Question 2: If yes, is it worth trying to find the limits when it 
>>>> goes too slow
>>>> and perhaps file a ticket?
>>>
>>>
>>> I hate to bring this up so late in the discussion, but have you tried
>>> -wo SKIP_NOSOURCE=YES?  When mosaicing small images into large images it
>>> prevents processing of chunks for which there is no source data.  
>>> Otherwise
>>> the whole output image is generated for each input image.
>>>
>>> There is an open ticket on this issue to apply this option by default.
>>>
>>> BTW, in combination with SKIP_NOSOURCE it can be helpful to keep the -wm
>>> option to a modest size to the tiles aren't too large.  A 200MB 
>>> should be
>>> sufficient - especially if the input and output image are tiled.
>>>
>>
>> Just to repeat my earlier observations: over the last days, I tried 
>> with all kinds of -wm and GDAL_CACHEMAX settings, and also added the 
>> options TILED=YES and SKIP_NOSOURCE=YES. I haven't documented the 
>> performance changes systematically, but my overall impression was that 
>> the changes were rather small. I also tried -multi on my dual 
>> processor machine, but I ended up with a fatal error.
>>
>> What really helped was to follow Felix' approach, who wrote in the 
>> first mail of this thread:
>>> Now, a second script, pasting first "slices" of 5*30° together,
>>> and than merging all the slices...
>>
>> This way, one can reduce the processing time to some 10-20% compared 
>> to the "all in one run approach". Concerning the size of the 
>> intermediate slices: the smaller they are, the faster it goes. The 
>> final step of gdalbuilding a vrt from the imtermediate slices and 
>> gdal_translating it to the final output.tif is also very fast.
>>
>> In order to avoid NODATA gaps in the final output.tif: It seems to be 
>> safer if the intermediate slices are slightly overlapping, rather 
>> being than just adjacent.
>>
>> Hermann
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>>