[gdal-dev] gdal_translate (3.1.0dev) "never" finishes on large jpeg cogs... REALLLLLY long time to unload.

[Jeremy Palmer]

Hi Andrew,

On Wed, Apr 22, 2020 at 6:11 AM Even Rouault <even.rouault at spatialys.com>
wrote:

> Andrew,
>
>
>
> > When I create a mask band in a large lzw-compressed or jpeg-compressed
> tif
>
> > using the COG driver it dramatically increases processing time over
> writing
>
> > RGBA (hours instead of minutes), so the issue is not jpeg compression,
> it's
>
> > the creation of the mask band. Steps to reproduce:
>
>
>
> There's clearly some complications & overhead internally to be able to
> deal with non-alpha mask bands. I wouldn't have expected it to be as large
> as you observed though. Would require deeper investigation to understand
> why that causes such performance difference.
>

I would also be interested to see what steps you have taken to process the
COG. We have used the driver to create both WEBP, and JPEG (including the
1bit transparency mask) COGs up to 200GB in size. It did take a couple of
days on a mid-sized AWS EC2 instance but completed OK. Note the most of the
time is taken up with the overview generation on large COGs. This will
hopefully improve soon with multi-threaded overview computation likely
coming to GDAL.

Warm Regards,
Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20200422/ec8010bf/attachment.html>