[gdal-dev] Creating Geotiff is slow

[Thomas Coquet]

Hello,

I am trying to convert a full JPEG2000 image (Sentinel 2) to a GeoTIFF
(10000x10000 pixels).

My config is the following: GDAL 3.1.0, released 2020/05/03, freshly
compiled on my computer (Ubuntu 18.04).

I use the following env variables:

OPJ_NUM_THREADS=8
GDAL_NUM_THREADS=1
GDAL_CACHEMAX=1000

The command line is:

gdal_translate -of GTiff -co COMPRESS=NONE -co TILED=YES
T15RTN_20190328T170201_B02.jp2 /mnt/ramdisk/test-openjpeg.tif

As stated in its name, /mnt/ramdisk is in memory, so there should not be
any I/O lag.

I observed the following pattern:
- decompression at full speed, using all my laptop cores (about 5sec);
- a single thread starts, to write the GeoTIFF (about 5sec as well).

I would have expected the GeoTIFF creation to be almost instantaneous given
the fact that there is no compression.

If I set GDAL_NUM_THREADS > 1, I do not manage to reach full CPU speed (all
cores are used, at about 50%), and it takes about 10 secs as well.

If I use mapbox/rasterio to read the raster into memory then to write it
back to a GeoTIFF file, the whole process takes 5s, the GeoTIFF write being
almost instantaneous.

I am puzzled by this behavior... Is there something wrong with my GDAL
settings ?
Is there a way to speed up the GeoTIFF creation ? Why is it faster using
python ?

Thank you!
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20200616/960992dc/attachment.html>