[gdal-dev] OpenJPEG Testers Needed

[Aaron Boxer]

So, it turns out to be quite complicated to re-factor the wavelet transform
to only transform a sub-region
inside a single tile image.

I've got it working now for lossy decode, lossless decode also works but 4
units tests are broken at the moment.

And, I am seeing a massive increase in performance: I tested an 11000 x
14000 lossless compressed single tile RGB 8 bit image,
decoding a 1000x1000 region in the middle of the image.  OpenJPEG master
takes around 60 seconds to decompress, and
my branch takes 3 seconds, on my 4 year old i7.. The cores are not even
half utilized, so this time should come down by at least a factor of two.

One remaining issue is memory allocation: still need to allocate gigabytes
of memory to do this. But, this should be an easy fix over the next few
weeks.

Will keep you all posted.

Cheers,
Aaron




On Wed, Jan 13, 2016 at 2:49 PM, Aaron Boxer <boxerab at gmail.com> wrote:

>
>
> On Wed, Jan 13, 2016 at 2:40 PM, Even Rouault <even.rouault at spatialys.com>
> wrote:
>
>>
>> > So, are you saying there are actual pixel differences between encoded
>> file
>> > and original?
>>
>> Yes
>>
>
> I see. So, this is repeatable every time you run the test suite?  And you
> can't reproduce
> with opj_compress followed by opj_decompress ?
>
>
>> > > oh, so how does one make a forward struct declaration to make gcc 4.4
>> > happy ?
>>
>> Apparently it doesn't like to see "typedef struct foo { ... } bar"
>> (openjpeg.h) and "typedef struct foo bar" (region_mgr.h).
>> If openjpeg.h was "struct foo { ... }", that would work (but we cannot
>> touch
>> that one). Why not just including openjpeg.h in region_mgr.h to avoid the
>> typedef redefinitions ?
>>
>>
> Thanks, I moves some code around and now it should compile on 4.4.
>
>
>> >
>> > > Testing with 2048x2048 tiled images, with OpenMP enabled, I can see
>> perf
>> > > improvements.
>> > > For example ~2s  (4 vCPU) vs ~3s (openjpeg 2.1) to decode a tile.
>> > >
>> > > Great. So, this is good. What are your system specs, may I ask ?
>>
>> Was on a i5 CPU 750 @ 2.67GHz, 4 GB RAM, 64bit build
>>
>> OK, thanks.
>
>
>> > >
>> > >
>> > >
>> > >
>> > > With a single tiled 11310x17310 image with which GDAL uses the
>> > > opj_set_decode_area() area,
>> > > I couldn't really see the difference as the memory consumption grows
>> to
>> > > above 2 GB,
>> > > which cause disk trashing on the system on which I test.
>> >
>> > Yes, that makes sense, there is still work to be done, I guess. The
>> wavelet
>> > transform
>> > needs to be modified to be more memory efficient.  What size of decode
>> area
>> > would you
>> > typically want to set on an image of that size ?
>>
>> One use case could be for displaying in a desktop GIS application (QGIS
>> for
>> example), so the size of the display area. Let's say 2000x1000 max.
>>
>>
> Great, thanks for the feedback.
>
>
>
>
>> >
>> >
>> > Thanks again,
>> > Aaron
>> >
>> >
>> > Even
>> >
>> > > --
>> > > Spatialys - Geospatial professional services
>> > > http://www.spatialys.com
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20160122/a9f5565f/attachment.html>