<div dir="ltr"><div><div><div><div> Greetings,<br><br></div>I am using the same files, I copied them from an AWS Bucket to a local AWS Instance.<br></div>I tried gdal_merge << tries to create 300GB file<br></div>I tried gdal_translate ran but created 2.5 GB not 6.9 GB file<br></div><div>Now trying gdalwarp.</div><div><br></div><div>the gdalinfo is the same in both datasets:</div><div><pre>coastal-2020.tif (6.9GB)</pre></div><div>Driver: GTiff/GeoTIFF<br>Size is 450000, 225000<br>Coordinate System is:<br>GEOGCRS["WGS 84",<br> DATUM["World Geodetic System 1984",<br> ELLIPSOID["WGS 84",6378137,298.257223563,<br> LENGTHUNIT["metre",1]]],<br> PRIMEM["Greenwich",0,<br> ANGLEUNIT["degree",0.0174532925199433]],<br> CS[ellipsoidal,2],<br> AXIS["geodetic latitude (Lat)",north,<br> ORDER[1],<br> ANGLEUNIT["degree",0.0174532925199433]],<br> AXIS["geodetic longitude (Lon)",east,<br> ORDER[2],<br> ANGLEUNIT["degree",0.0174532925199433]],<br> ID["EPSG",4326]]<br>Data axis to CRS axis mapping: 2,1<br>Origin = (-180.000000000000000,90.000000000000000)<br>Pixel Size = (0.000800000000000,-0.000800000000000)<br>Metadata:<br> AREA_OR_POINT=Area<br> datetime_created=2022-11-14 18:05:14.053301<br>Image Structure Metadata:<br> COMPRESSION=LZW<br> INTERLEAVE=BAND<br> PREDICTOR=3<br>Corner Coordinates:<br>Upper Left (-180.0000000, 90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"N)<br>Lower Left (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)<br>Upper Right ( 180.0000000, 90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"N)<br>Lower Right ( 180.0000000, -90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"S)<br>Center ( 0.0000000, 0.0000000) ( 0d 0' 0.01"E, 0d 0' 0.01"N)<br>Band 1 Block=128x128 Type=Float32, ColorInterp=Gray<br> Description = score<br> NoData Value=-9999<br>Band 2 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = severity_value<br> NoData Value=-9999<br>Band 3 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = severity_min<br> NoData Value=-9999<br>Band 4 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = severity_max<br> NoData Value=-9999<br>Band 5 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = likelihood<br> NoData Value=-9999<br>Band 6 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = return_time<br> NoData Value=-9999<br>Band 7 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = likelihood_confidence<br> NoData Value=-9999<br>Band 8 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = climate_reliability<br> NoData Value=-9999<br>Band 9 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = hazard_reliability<br> NoData Value=-9999</div><div><div><div><br><pre>5_UK_coastal-2020.tif (600MB)</pre><div><div>Driver: GTiff/GeoTIFF<br>Size is 450000, 225000<br>Coordinate System is:<br>GEOGCRS["WGS 84",<br> DATUM["World Geodetic System 1984",<br> ELLIPSOID["WGS 84",6378137,298.257223563,<br> LENGTHUNIT["metre",1]]],<br> PRIMEM["Greenwich",0,<br> ANGLEUNIT["degree",0.0174532925199433]],<br> CS[ellipsoidal,2],<br> AXIS["geodetic latitude (Lat)",north,<br> ORDER[1],<br> ANGLEUNIT["degree",0.0174532925199433]],<br> AXIS["geodetic longitude (Lon)",east,<br> ORDER[2],<br> ANGLEUNIT["degree",0.0174532925199433]],<br> ID["EPSG",4326]]<br>Data axis to CRS axis mapping: 2,1<br>Origin = (-180.000000000000000,90.000000000000000)<br>Pixel Size = (0.000800000000000,-0.000800000000000)<br>Metadata:<br> AREA_OR_POINT=Area<br> datetime_created=2022-11-14 18:05:14.053301<br> hostname=posix.uname_result(sysname='Linux', nodename='ip-172-31-12-125', release='5.15.0-1022-aws', version='#26~20.04.1-Ubuntu SMP Sat Oct 15 03:22:07 UTC 2022', machine='x86_64')<br>Image Structure Metadata:<br> COMPRESSION=LZW<br> INTERLEAVE=BAND<br> PREDICTOR=3<br>Corner Coordinates:<br>Upper Left (-180.0000000, 90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"N)<br>Lower Left (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)<br>Upper Right ( 180.0000000, 90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"N)<br>Lower Right ( 180.0000000, -90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"S)<br>Center ( 0.0000000, 0.0000000) ( 0d 0' 0.01"E, 0d 0' 0.01"N)<br>Band 1 Block=128x128 Type=Float32, ColorInterp=Gray<br> Description = score<br> NoData Value=-9999<br>Band 2 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = severity_value<br> NoData Value=-9999<br>Band 3 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = severity_min<br> NoData Value=-9999<br>Band 4 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = severity_max<br> NoData Value=-9999<br>Band 5 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = likelihood<br> NoData Value=-9999<br>Band 6 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = return_time<br> NoData Value=-9999<br>Band 7 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = likelihood_confidence<br> NoData Value=-9999<br>Band 8 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = climate_reliability<br> NoData Value=-9999<br>Band 9 Block=128x128 Type=Float32, ColorInterp=Undefined<br> Description = hazard_reliability<br> NoData Value=-9999<br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><p style="margin:0cm 0cm 0pt"><span style="font-family:Calibri;font-size:8pt"> Regards,</span></p><p style="margin:0cm 0cm 0pt"><span style="font-family:Calibri;font-size:8pt"><br></span></p><p style="margin:0cm 0cm 0pt"><span style="font-family:Calibri;font-size:8pt">Clive Swan</span></p>
<p style="margin:0cm 0cm 0pt"><span><font size="3"><font face="Calibri">--</font></font></span></p><pre>Hi,
If you are still struggling with the same old problem could you please finally send the gdalinfo reports of your two input files which are this time:
coastal-2020.tif
5_UK_coastal-2020.tif
-Jukka Rahkonen-
Lähettäjä: gdal-dev <<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">gdal-dev-bounces at lists.osgeo.org</a>> Puolesta Clive Swan
Lähetetty: tiistai 13. joulukuuta 2022 17.23
Vastaanottaja: <a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">gdal-dev at lists.osgeo.org</a>
Aihe: [gdal-dev] gdalwarp running very slow
Greetings,
I am running gdalwarp on a 6GB (output) and 600MB (input) tif image, the AWS Instance has approx 60 VCPU
It has taken over 6 hours so far - still running, is it possible to optimise this and speed it up??
gdalwarp -r near -overwrite coastal-2020.tif 5_UK_coastal-2020.tif -co BIGTIFF=YES -co COMPRESS=LZW -co BLOCKXSIZE=128 -co BLOCKYSIZE=128 -co NUM_THREADS=ALL_CPUS --config CPL_VSIL_USE_TEMP_FILE_FOR_RANDOM_WRITE YES</pre><p style="margin:0cm 0cm 0pt"><span><font size="3"><font face="Calibri"></font></font></span></p></div></div></div></div></div></div></div></div></div></div></div></div>