[gdal-dev] More info on gdalwarp resampling methods?

[Matt.Wilkie at yukon.ca]

Issues opened:


·         #3630 gdalwarp -multi should auto-enable GDAL_NUM_THREADS=ALL_CPUS <https://github.com/OSGeo/gdal/issues/3630>

·         #3631 gdalwarp error "Failed to acquire WarpMutex in WarpRegion()"<https://github.com/OSGeo/gdal/issues/3631>

·         #3632 gdalwarp mode resample much more expensive than needed<https://github.com/OSGeo/gdal/issues/3632>

-Matt

From: Even Rouault <even.rouault at spatialys.com>
Sent: March 31, 2021 10:27 AM
To: Matt.Wilkie <Matt.Wilkie at yukon.ca>; gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] More info on gdalwarp resampling methods?


*** External email: Do not click on links or attachments except from trusted senders. ***

******************************************************************************************




Matt,

I see that in the -multi mode, the I/O threads waits for a maximum of 10 minutes for the computation thread to have finished. And you likely hit that limit because of the large value of -wm you set. There's no reason we max at 10 minutes however. Please find a but about that, including my analysis.

For better speed, I'd suggest you add --config GDAL_NUM_THREADS ALL_CPUS (or a given number of threads). The -multi switch only does one thread for I/O and one thread for computation. By adding --config GDAL_NUM_THREADS xxx you'll also multi-thread the computation part. The gain of -multi alone is generally neglectable. We should probably make -multi turn on --config GDAL_NUM_THREADS ALL_CPUS as well.

I also see that the mode resampling spend an awful lot of time on line "memset(panVals, 0, nBins*sizeof(int));" where nBins=65536 on Int16 data, and that for each output pixel. We could probably do something much less expensive for most cases. Please file a separate enhancement ticket about that particular case.

Even


Le 31/03/2021 à 18:46, Matt.Wilkie at yukon.ca<mailto:Matt.Wilkie at yukon.ca> a écrit :
gdalwarp --config GDAL_CACHEMAX 4096 -wm 512 -multi -co TILED=YES -co COMPRESS=ZSTD -t_srs EPSG:3579 -r mode -co predictor=2 tri_mergedDEM_clipped_16bit.tif tri_mergedDEM_clipped_16bit_ytalb-rs-mode.tif

--

http://www.spatialys.com<https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2fwww.spatialys.com&umid=682B3DEE-BED8-6F05-8639-74A30E43DC88&auth=c132af8ee7c9d1278d61a701569070a095ce962e-453104e8853fb736505949aed5860b4e74f3ecf8>

My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210331/937aff36/attachment-0001.html>