[gdal-dev] OpenJPEG Testers Needed

[Even Rouault]

Le mercredi 13 janvier 2016 18:10:52, Aaron Boxer a écrit :
> Dear GDAL users,
> 
> As I've mentioned previously, there are big changes coming to the OpenJPEG
> library. Performance is going up and memory usage is going down, by
> significant amounts.
> 
> To do all of this, we have had to make significant changes to the core of
> the library. So, even though all of our unit tests are passing, we would
> still like to know more about how the new code works in a geospatial work
> flow: is it stable, how much faster is it, how has memory usage changes?
> 
> If anyone has a few spare cycles for testing this out, it would be very
> much appreciated!
> 
> I am maintaining the new branch on my github repo, so you can go here
> for the latest code if you are interested:
> 
> https://github.com/CodecCentral/openjpeg/tree/decode_region
> 
> By the way, this branch also includes another new feature: fast decode of
> sub-window inside of a tile.  This is done using the existing openjpeg -d
> command line flag. For large single tile images, this allows users to
> quickly view a sub region of the image.
> 

Some feedback:

I've just tested the decode_region branch against the test suite of the JP2OpenJPEG driver
( https://svn.osgeo.org/gdal/trunk/autotest/gdrivers/jp2openjpeg.py )
and noticed a regression with respect to lossless encoding:

From autotest/gdrivers directory :

official openjpeg 2.1 (or master branch) :
$ gdal_translate data/byte.jp2 out.jp2 -of jp2openjpeg -co quality=100 -co reversible=yes

decode_region branch:
$  LD_LIBRARY_PATH=/home/even/tmp/openjpeg/install/lib:$LD_LIBRARY_PATH gdal_translate data/byte.jp2 out2.jp2 -of jp2openjpeg -co quality=100 -co reversible=yes

Official openjpeg 2.1 (or master branch) outputs give same pixel values as the source image :

$ python ../../gdal/swig/python/scripts/gdalcompare.py out.jp2 data/byte.jp2
Files differ at the binary level.
Differences Found: 1

==> ok: they have same pixel value, the binary differences comes from not using the same encoder

But they are tiny differences with decode_region branch

$ python ../../gdal/swig/python/scripts/gdalcompare.py out.jp2 out2.jp2
Files differ at the binary level.
Band 1 checksum difference:
  Golden: 50054
  New:    49943
  Pixels Differing: 833
  Maximum Pixel Difference: 1.0
Differences Found: 2

The other tests pass

I cannot reproduce directly with opj_compress however.


(side node: there's a build issue with old gcc 4.4 : 
In file included from /home/even/tmp/openjpeg/src/lib/openjp2/opj_includes.h:194,
                 from /home/even/tmp/openjpeg/src/lib/openjp2/bio.c:38:
/home/even/tmp/openjpeg/src/lib/openjp2/region_mgr.h:35: error: redefinition of typedef ‘opj_image_t’
/home/even/tmp/openjpeg/src/lib/openjp2/openjpeg.h:686: note: previous declaration of ‘opj_image_t’ was here
In file included from /home/even/tmp/openjpeg/src/lib/openjp2/opj_includes.h:197,
                 from /home/even/tmp/openjpeg/src/lib/openjp2/bio.c:38:
/home/even/tmp/openjpeg/src/lib/openjp2/tcd.h:185: error: redefinition of typedef ‘opj_tcd_tile_t’
/home/even/tmp/openjpeg/src/lib/openjp2/region_mgr.h:36: note: previous declaration of ‘opj_tcd_tile_t’ was here
make[2]: *** [src/lib/openjp2/CMakeFiles/openjp2.dir/bio.c.o] Erreur 1
)


Testing with 2048x2048 tiled images, with OpenMP enabled, I can see perf improvements.
For example ~2s  (4 vCPU) vs ~3s (openjpeg 2.1) to decode a tile.

With a single tiled 11310x17310 image with which GDAL uses the opj_set_decode_area() area,
I couldn't really see the difference as the memory consumption grows to above 2 GB,
which cause disk trashing on the system on which I test.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com