<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.Shkpostityyli18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-ligatures:none;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 70.85pt 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">A better reference file can be created by materializing the VRT. The one created by gdal_create has the same raster values on all the pixels but the materialized one contains the same
original image data. Maybe in your test case with uncompressed outputs the difference is not so big, but please test it anyway.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">gdal_translate -of GTiff -co tiled=yes -co compress=LZW input.vrt output.tif<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">gdaladdo -ro output.tif<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">What is the difference in speed now? Ten times slower performance with vrt feels quite a lot.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Lähettäjä:</b> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>Puolesta </b>Denis Rykov via gdal-dev<br>
<b>Lähetetty:</b> keskiviikko 5. kesäkuuta 2024 4.36<br>
<b>Vastaanottaja:</b> gdal dev <gdal-dev@lists.osgeo.org><br>
<b>Aihe:</b> [gdal-dev] gdaladdo slowness for VRTs<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I spotted a slow calculation of overviews and I'm wondering what could be the reason.<br>
<br>
<span style="font-family:"Courier New"">$ gdal_create in.tif -if 20240602_230818_SN26_RR_VISUAL_MS.vrt</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">$ time gdaladdo -ro in.tif </span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-family:"Courier New"">gdaladdo -ro in.tif 4,25s user 6,23s system 34% cpu 30,623 total</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">But on the file of the same size and num of bands but VRT it takes much more time:<br>
<br>
<span style="font-family:"Courier New"">$ time gdaladdo -ro 20240602_230818_SN26_RR_VISUAL_MS.vrt<br>
gdaladdo -ro 55,36s user 5,06s system 44% cpu 2:14,79 total</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Here is the output of gdalinfo of the VRT:<br>
<br>
<span style="font-family:"Courier New"">$ gdalinfo 20240602_230818_SN26_RR_VISUAL_MS.vrt<br>
Driver: VRT/Virtual Raster<br>
Files: 20240602_230818_SN26_RR_VISUAL_MS.vrt<br>
20240602_230818_SN26_RR_VISUAL_MS_340_5020.tif<br>
20240602_230818_SN26_RR_VISUAL_MS_340_5040.tif<br>
20240602_230818_SN26_RR_VISUAL_MS_360_5020.tif<br>
20240602_230818_SN26_RR_VISUAL_MS_360_5040.tif<br>
20240602_230818_SN26_RR_VISUAL_MS_360_5060.tif<br>
Size is 20779, 46754<br>
Coordinate System is:<br>
PROJCRS["WGS 84 / UTM zone 59S",<br>
BASEGEOGCRS["WGS 84",<br>
DATUM["World Geodetic System 1984",<br>
ELLIPSOID["WGS 84",6378137,298.257223563,<br>
LENGTHUNIT["metre",1]]],<br>
PRIMEM["Greenwich",0,<br>
ANGLEUNIT["degree",0.0174532925199433]],<br>
ID["EPSG",4326]],<br>
CONVERSION["UTM zone 59S",<br>
METHOD["Transverse Mercator",<br>
ID["EPSG",9807]],<br>
PARAMETER["Latitude of natural origin",0,<br>
ANGLEUNIT["degree",0.0174532925199433],<br>
ID["EPSG",8801]],<br>
PARAMETER["Longitude of natural origin",171,<br>
ANGLEUNIT["degree",0.0174532925199433],<br>
ID["EPSG",8802]],<br>
PARAMETER["Scale factor at natural origin",0.9996,<br>
SCALEUNIT["unity",1],<br>
ID["EPSG",8805]],<br>
PARAMETER["False easting",500000,<br>
LENGTHUNIT["metre",1],<br>
ID["EPSG",8806]],<br>
PARAMETER["False northing",10000000,<br>
LENGTHUNIT["metre",1],<br>
ID["EPSG",8807]]],<br>
CS[Cartesian,2],<br>
AXIS["easting",east,<br>
ORDER[1],<br>
LENGTHUNIT["metre",1]],<br>
AXIS["northing",north,<br>
ORDER[2],<br>
LENGTHUNIT["metre",1]],<br>
ID["EPSG",32759]]<br>
Data axis to CRS axis mapping: 1,2<br>
Origin = (353999.799999999988358,5068000.629999999888241)<br>
Pixel Size = (0.770000000000000,-0.770000000000000)<br>
Corner Coordinates:<br>
Upper Left ( 353999.800, 5068000.630) (169d 9'45.50"E, 44d31'35.55"S)<br>
Lower Left ( 353999.800, 5032000.050) (169d 9' 8.52"E, 44d51' 1.68"S)<br>
Upper Right ( 369999.630, 5068000.630) (169d21'50.06"E, 44d31'46.58"S)<br>
Lower Right ( 369999.630, 5032000.050) (169d21'17.12"E, 44d51'12.83"S)<br>
Center ( 361999.715, 5050000.340) (169d15'30.36"E, 44d41'24.33"S)<br>
Band 1 Block=128x128 Type=Byte, ColorInterp=Red<br>
NoData Value=0<br>
Band 2 Block=128x128 Type=Byte, ColorInterp=Green<br>
NoData Value=0<br>
Band 3 Block=128x128 Type=Byte, ColorInterp=Blue<br>
NoData Value=0<br>
Band 4 Block=128x128 Type=Byte, ColorInterp=Undefined<br>
NoData Value=0</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">And gdalinfo output for one of the underlying rasters:<br>
<br>
gdalinfo 20240602_230818_SN26_RR_VISUAL_MS_360_5040.tif<br>
Driver: GTiff/GeoTIFF<br>
Files: 20240602_230818_SN26_RR_VISUAL_MS_360_5040.tif<br>
Size is 10390, 25974<br>
Coordinate System is:<br>
PROJCRS["WGS 84 / UTM zone 59S",<br>
BASEGEOGCRS["WGS 84",<br>
DATUM["World Geodetic System 1984",<br>
ELLIPSOID["WGS 84",6378137,298.257223563,<br>
LENGTHUNIT["metre",1]]],<br>
PRIMEM["Greenwich",0,<br>
ANGLEUNIT["degree",0.0174532925199433]],<br>
ID["EPSG",4326]],<br>
CONVERSION["UTM zone 59S",<br>
METHOD["Transverse Mercator",<br>
ID["EPSG",9807]],<br>
PARAMETER["Latitude of natural origin",0,<br>
ANGLEUNIT["degree",0.0174532925199433],<br>
ID["EPSG",8801]],<br>
PARAMETER["Longitude of natural origin",171,<br>
ANGLEUNIT["degree",0.0174532925199433],<br>
ID["EPSG",8802]],<br>
PARAMETER["Scale factor at natural origin",0.9996,<br>
SCALEUNIT["unity",1],<br>
ID["EPSG",8805]],<br>
PARAMETER["False easting",500000,<br>
LENGTHUNIT["metre",1],<br>
ID["EPSG",8806]],<br>
PARAMETER["False northing",10000000,<br>
LENGTHUNIT["metre",1],<br>
ID["EPSG",8807]]],<br>
CS[Cartesian,2],<br>
AXIS["(E)",east,<br>
ORDER[1],<br>
LENGTHUNIT["metre",1]],<br>
AXIS["(N)",north,<br>
ORDER[2],<br>
LENGTHUNIT["metre",1]],<br>
USAGE[<br>
SCOPE["Engineering survey, topographic mapping."],<br>
AREA["Between 168°E and 174°E, southern hemisphere between 80°S and equator, onshore and offshore. New Zealand."],<br>
BBOX[-80,168,0,174]],<br>
ID["EPSG",32759]]<br>
Data axis to CRS axis mapping: 1,2<br>
Origin = (359999.640000000013970,5060000.330000000074506)<br>
Pixel Size = (0.770000000000000,-0.770000000000000)<br>
Metadata:<br>
AREA_OR_POINT=Area<br>
Image Structure Metadata:<br>
COMPRESSION=LZW<br>
INTERLEAVE=PIXEL<br>
Corner Coordinates:<br>
Upper Left ( 359999.640, 5060000.330) (169d14' 9.36"E, 44d35'59.00"S)<br>
Lower Left ( 359999.640, 5040000.350) (169d13'49.66"E, 44d46'46.86"S)<br>
Upper Right ( 367999.940, 5060000.330) (169d20'12.11"E, 44d36' 4.44"S)<br>
Lower Right ( 367999.940, 5040000.350) (169d19'53.53"E, 44d46'52.34"S)<br>
Center ( 363999.790, 5050000.340) (169d17' 1.19"E, 44d41'25.70"S)<br>
Band 1 Block=256x256 Type=Byte, ColorInterp=Red<br>
NoData Value=0<br>
Band 2 Block=256x256 Type=Byte, ColorInterp=Green<br>
NoData Value=0<br>
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue<br>
NoData Value=0<br>
Band 4 Block=256x256 Type=Byte, ColorInterp=Undefined<br>
NoData Value=0<br>
</span><br>
What could be the reason for such a significant difference?<o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>