<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
This is a follow-up on <a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/issues/14063">https://github.com/OSGeo/gdal/issues/14063</a>.<br>
<br>
I decided to take it to the list instead, because I'm not sure
whether this is an issue or simply a discussion on performance. (Let
me know if you rather take this to gh issues again; happy to do so).<br>
<br>
Please read the original issue first, to avoid wasting time on
questions that were already answered there :-)<br>
<br>
Short context: trying to get good performance out of a GTI mosaic
which wraps (potentially a lot of) remote tifs (on AWS S3). The tifs
are in UTM, the mosaic is reprojecting to epgs:4326.<br>
<br>
After the performance optimizations made by Even as response to
#14063 (<a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/pull/14089">https://github.com/OSGeo/gdal/pull/14089</a> and
<a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/pull/14094">https://github.com/OSGeo/gdal/pull/14094</a>), I did some testing. The
results were rather... confusing.<br>
<br>
All these tests were done with a fresh gdal master build (GDAL
3.13.0dev-28b33d81c7, released 2026/03/15).<br>
<br>
What I expected (hoped):<br>
- wrapping a (single) remote tif with a GTI should not affect
performance too much<br>
- gdal_translate to have better or similar performance on the GTI
than gdalwarp, especially with using -oo WARPING_MEMORY_SIZE. As
Even said, using gdalwarp means the warping logic is used twice,
where it is only needed once. Using gdal_translate should therefore
show better performance.<br>
<br>
What I found:<br>
- gdalwarp on the GTI is about 50% slower than on the remote tiff
directly (but maybe that is simply the cost of having a GTI wrapper?
I hoped it would be less)<br>
- gdal_translate is still waaay slower (10x) than gdal_warp. Using
WARPING_MEMORY_SIZE hardly seems to make a difference.<br>
- this is only/mainly when targeting remote tifs. When using local
files, the timings are very different. But: the remote case is the
case we need, and what want to optimize. So I focus on that.<br>
<br>
My tests were on a 22-core (according to "nproc") machine, 64G ram.
Results will probably vary with number of cores, network speed,
physical location (I'm in Europe, these tifs are probably in a
bucket in the US), and also the size of the block extracted. All
tests below extract the same 0.2 degree block.<br>
<br>
These are the commands I ran as test:<br>
<br>
# export config options<br>
export GDAL_CACHEMAX=1GB GDAL_NUM_THREADS=ALL_CPUS
DISABLE_READDIR_ON_OPEN=EMPTY_DIR
CPL_VSIL_CURL_ALLOWED_EXTENSIONS="tiff" AWS_NO_SIGN_REQUEST=YES<br>
<br>
# create a GTI wrapping 1 remote tif<br>
gdal driver gti create --resolution 0.00006,0.00006 --ot Int8
--band-count 64 --nodata -128 --dst-crs epsg:4326 --of gpkg
/vsis3/us-west-2.opendata.source.coop/tge-labs/aef/v1/annual/2024/30N/xpzba7dllw4la2007-0000008192-0000000000.tiff
test_1tiff.gti.gpkg<br>
<br>
# start of tests<br>
# each test extracts the same 0.2 x 0.2 degree block.<br>
<br>
# gdal_warp on the remote tif directly, doing the exact same
reprojecting as the GTI is doing<br>
gdalwarp -overwrite -te -5.8 5.6 -5.6 5.8 -tr 0.00006 -0.00006
-t_srs epsg:4326 -wm 1G -co tiled=yes -co blockxsize=512 -co
blockysize=512 -co interleave=band
/vsis3/us-west-2.opendata.source.coop/tge-labs/aef/v1/annual/2024/30N/xpzba7dllw4la2007-0000008192-0000000000.tiff
result_tiff_allbands.tif<br>
# time: 0m33s<br>
<br>
# gdalwarp on the GTI which wraps the same tif<br>
gdalwarp -overwrite -te -5.8 5.6 -5.6 5.8 -wm 1G -co tiled=yes -co
blockxsize=512 -co blockysize=512 -co interleave=band
test_1tiff.gti.gpkg result_gti_allbands.tif<br>
# time: 0m47s<br>
<br>
# gdal_translate on the GTI with -oo WARPING_MEMORY_SIZE set to 1G<br>
gdal_translate -oo WARPING_MEMORY_SIZE=1GB -projwin -5.8 5.8 -5.6
5.6 -co tiled=yes -co blockxsize=512 -co blockysize=512 -co
interleave=band test_1tiff.gti.gpkg gt_result_gti_allbands.tif<br>
# time: 5m31s<br>
<br>
# gdal_translate on the GTI without the -oo WARPING_MEMORY_SIZE
option<br>
gdal_translate -projwin -5.8 5.8 -5.6 5.6 -co tiled=yes -co
blockxsize=512 -co blockysize=512 -co interleave=band
test_1tiff.gti.gpkg gt_result_gti_allbands.tif<br>
# time: 5m16s<br>
<br>
<div class="moz-signature">-- <br>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title></title>
<div class="moz-signature">
<div class="moz-signature">
<div dir="ltr" style="margin-left:2.88pt;" align="left"><br>
<table
style="border: medium none; border-collapse: collapse;"
width="408" height="179">
<colgroup><col width="60"><col width="283"></colgroup><tbody>
<tr style="height:10.799999999999999pt">
<td
style="border-left:solid #003c5a
1.5pt;border-right:solid #ffffff
1pt;border-bottom:solid #ffffff 1pt;border-top:solid
#ffffff 1pt;vertical-align:top;padding:4.32pt 4.32pt
4.32pt
4.32pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"><span
style="border:none;display:inline-block;overflow:hidden;width:53px;height:53px;"><img
src="https://lh6.googleusercontent.com/t1GD8ftw5REdi0_kM1UjrXeX-Y-jF5xKxDqUGLsLKzftVahqw_CWQhqciHkLLYkldd9pcYmNd-pmzNnY1fc93vYpJ0Q0P-O1VC51jsTJUy-yhjF4ioK2hGq156_JYVROec9im6tW"
style="margin-left:0px;margin-top:0px;" width="53"
height="53"></span></span></p>
</td>
<td
style="border-left:solid #ffffff
1pt;border-right:solid #ffffff 1pt;border-bottom:solid
#ffffff 1pt;border-top:solid #ffffff
1pt;vertical-align:middle;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#003c5a;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Vincent Schut</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:10pt;font-family:Arial;color:#003c5a;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Remote Sensing Software Engineer
</span></p>
</td>
</tr>
<tr style="height:10.799999999999999pt">
<td colspan="2"
style="border-left:solid #003c5a
1.5pt;border-right:solid #ffffff
1pt;border-bottom:solid #ffffff 1pt;border-top:solid
#ffffff 1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:8pt;font-family:Arial;color:#b7b7b7;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"><span
style="border:none;display:inline-block;overflow:hidden;width:217px;height:34px;"><img
src="https://lh5.googleusercontent.com/H6PdQi9jrkkJRWy7wGQ92waInTqhh3atodO0R0CNgKm2YooJBK2vOEU2M7t5HB5FMJyOUhyFxsTqtEnqLi3zXnRbPNJEvoEyr9RyBFKKHUjT8py8Pu92ga3Urmp_zARoHqtNFY6K"
style="margin-left:0px;margin-top:0px;"
width="217" height="34"></span></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"
id="docs-internal-guid-30036d47-7fff-c8bb-a5a1-95628900a5a0"><span
style="font-size:8pt;font-family:Arial;color:#b7b7b7;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">+31 302272679 ~ Maliebaan 22 | 3581CP | Utrecht | Netherlands</span></p>
<a
href="https://www.linkedin.com/company/satelligence/"
style="text-decoration:none;"
id="docs-internal-guid-fa359a62-7fff-0882-e450-fae9010e52d8"><span
style="font-size:8pt;font-family:Arial;color:#b7b7b7;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;-webkit-text-decoration-skip:none;text-decoration-skip-ink:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Linkedin</span></a><span
style="font-size:8pt;font-family:Arial;color:#b7b7b7;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> ~ </span><a
href="http://www.satelligence.com"
style="text-decoration:none;"><span
style="font-size:8pt;font-family:Arial;color:#b7b7b7;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;-webkit-text-decoration-skip:none;text-decoration-skip-ink:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">satelligence.com</span></a><a
href="http://www.satelligence.com"
style="text-decoration:none;"><span
style="font-size:8pt;font-family:Arial;color:#b7b7b7;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;-webkit-text-decoration-skip:none;text-decoration-skip-ink:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"></span></a></td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</body>
</html>