[gdal-dev] Problems generating a COG file

[Even Rouault]

Javier,

- which GDAL version it is ?

- the "corrupted double-linked list" is definitely a proof of either a 
bug, or a mis-configuration. This is a message we see sometimes when 
GDAL accidentally links against 2 different libproj. Can you check "ldd 
/path/to/libgdal.so" to see if there is not 2 libproj appearing ?

- can you reproduce the issue with a fully blank file with the same 
characterists. For example try "gdal_create -if web_mercator.tif 
web_mercator_blank.tif -co SPARSE_OK=YES -co TILED=YES", and then run 
the gdal_translate to COG on that web_mercator_blank.tif

- does running in single threaded mode with GDAL_NUM_THREADS=1 make a 
difference regarding the error messages?

Even

Le 20/08/2025 à 19:17, Javier Jimenez Shaw via gdal-dev a écrit :
> Hi
>
> I am warping a file to EPSG:3857 and later generating a COG with JPEG 
> compression.
> The input tif file has also JPEG compression. (I learned that I need 
> the -dstalpha to keep it transparent)
> I am doing it in Ubuntu 22.04 using conda.
>
> First I tried this, using LZW as output of the warp
>
> GDAL_CACHEMAX=16GB GDAL_NUM_THREADS=ALL_CPUS gdalwarp -t_srs EPSG:3857 
> -ct "+proj=pipeline +step +inv +proj=tmerc +lat_0=49 +lon_0=-2 
> +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +step +proj=push 
> +v_3 +step +proj=cart +ellps=airy +step +proj=helmert +x=446.448 
> +y=-125.157 +z=542.06 +rx=0.15 +ry=0.247 +rz=0.842 +s=-20.489 
> +convention=position_vector +step +inv +proj=cart +ellps=WGS84 +step 
> +proj=pop +v_3 +step +proj=webmerc +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 
> +ellps=WGS84" -co COMPRESS=LZW -co BIGTIFF=YES -co TILED=YES -dstalpha 
> orthomosaic.tiff web_mercator.tif
>
> Creating output file that is 677881P x 303664L.
> Processing orthomosaic.tiff [1/1] : 
> 0...10...20...30...40...50...60...70...80...90...100 - done in 00:27:04.
>
> then I try to generate the COG
> GDAL_CACHEMAX=16GB  GDAL_NUM_THREADS=ALL_CPUS gdal_translate 
> web_mercator.tif web_mercator_cog.tif -co COMPRESS=JPEG -co 
> BIGTIFF=YES -of COG
> Input file size is 677881, 303664
> 0ERROR 1: LZWDecode:Not enough data at scanline 0 (short 40448 bytes)
> 0.                                                   - estimated 
> remaining time: 00:14:34
>
> and failed
>
> As it is an LZW problem, I tried with DEFLATE:
>
> GDAL_CACHEMAX=16GB GDAL_NUM_THREADS=ALL_CPUS gdalwarp -t_srs EPSG:3857 
> -ct "+proj=pipeline +step +inv +proj=tmerc +lat_0=49 +lon_0=-2 
> +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +step +proj=push 
> +v_3 +step +proj=cart +ellps=airy +step +proj=helmert +x=446.448 
> +y=-125.157 +z=542.06 +rx=0.15 +ry=0.247 +rz=0.842 +s=-20.489 
> +convention=position_vector +step +inv +proj=cart +ellps=WGS84 +step 
> +proj=pop +v_3 +step +proj=webmerc +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 
> +ellps=WGS84" -co COMPRESS=DEFLATE -co BIGTIFF=YES -co TILED=YES 
> -dstalpha orthomosaic.tiff web_mercator_d.tif
>
> Creating output file that is 677881P x 303664L.
> Processing orthomosaic.tiff [1/1] : 
> 0...10...20...30...40...50...60...70...80...90...100 - done in 00:27:34.
>
> And then the same
> GDAL_CACHEMAX=16GB  GDAL_NUM_THREADS=ALL_CPUS gdal_translate 
> web_mercator_d.tif web_mercator_cog.tif -co COMPRESS=JPEG -co 
> BIGTIFF=YES -of COG
> Input file size is 677881, 303664
> 0..                                                  - estimated 
> remaining time: 02:29:49ERROR 1: ZIPDecode:Decoding error at scanline 0
> 0...10...                              - estimated remaining time: 
> 00:42:12
>
> and failed :(
>
> Trying again I got this output at a similar point (but nothing about 
> ZIPDecode:Decoding error at scanline 0).
> corrupted double-linked list
> Aborted (core dumped)
>
> What can it be? Is there any workaround?
> The file is big in pixels, but a big part is transparent. 
> web_mercator_d.tif is "only" 14GB
>
> Thank you.
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250820/90b52e4a/attachment.htm>