[gdal-dev] Fwd: OpenJPEG

Aaron Boxer boxerab at gmail.com
Mon Dec 21 08:15:29 PST 2015

---------- Forwarded message ----------
From: Aaron Boxer <boxerab at gmail.com>
Date: Mon, Dec 21, 2015 at 10:27 AM
Subject: Re: [gdal-dev] OpenJPEG
To: Julien Malik <julien.malik at c-s.fr>

Hi Julien,

> Decoding at preccints level in OpenJPEG would definitely be a great
> addition.

Yes, sounds useful.

> I believe OpenJPEG performance could also be improved if multi-threading
> was managed directly inside the lib, since in the current situation
> OpenJPEG is purely monothreaded.
> From my experience, Kakadu does a great job in keeping all your cores
> occupied close to 100% during decoding, even when decoding a single tile
> (so this is probably linked to the capability to manage independent
> decoding at the precinct level).
> In 3 geospatial software I know leverage on OpenJPEG (gdal, orfeo toolbox,
> and snap) to decode big datasets (think tens or hundreds of thousand pixels
> in each dimension), multithreading is handled from the user side, by
> creating a number of independent contexts and run them in separate threads
> to improve performance.
> This means we are all interpreting the codestreams and seeking into them
> independently, which clearly sounds suboptimal.
> I suspect managing multi-threading at a lower level directly inside
> OpenJPEG could open the doors to much better resource management (cores
> occupation, reduced I/O, etc...).

The pull request I mentioned does use all cores for the computationally
intensive part of decoding. Reading through the file is still single
threaded. And this can take a good chunk
of time.

> I certainly concur with Even with the "as fast as it can be" :)
> Any percent you gain has a direct visible impact when (de)coding huge
> datasets.
Thanks for the feedback. Good news for OpenJPEG performance : they will be
*hiring* a developer next year to focus solely on performance.
If you guys are interested, please check out the OpenJPEG mailing list and
you can apply.

Also, I would encourage submitting all of these great ideas as issues on
the openjpeg github site:



On 20/12/2015 11:00, Even Rouault wrote:

> Aaron,
> I am working on some performance enhancements for OpenJPEG,
>> and I am curious about people's experience using the OpenJPEG driver with
>> GDAL:
>> Does it have all the features you need, if not what is missing?
> Being the developer of the driver, I believe it must be close to using the
> available capabilities of OpenJPEG as best as possible. I may have
> overlooked
> some features, so other eyes checking the code are always welcome.
>  From my point of view what is mainly missing in OpenJPEG (the lib):
> - ability to decode efficiently sub-windows inside a tile (I guess this
> means
> using precincts). This is particuarly true for single-tiled big files
> - reduce memory consumption. I've found that to decode a 2048x2048 tile of
> a
> Sentinel2 product, it takes 600 MB of memory whereas the uncompressed size
> of
> the tile is 8 MB (2048x2048*sizeof(int16)).
> - ability to open files who have more than 2 gigapixels (or 4, I'm not sure
> where the limit is)
> - currently the driver closes and reopen the file each time it reads a
> tile,
> because there are(were?) some instabilities in seeking through arbitrary
> tiles
> (ie reading potentially not in ascending tile order). Would be nice to be
> able
> to avoid that
> Given that it is the slowest J2K library around, what would be acceptable
>> performance in the geo-spatial world?
> As fast as possible :-)
> These changes apply to the current master version of OpenJPEG, which will
>> be released shortly.
> Were you refering to this PR :
> https://github.com/uclouvain/openjpeg/pull/668
> ?
> Even
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20151221/44eebe94/attachment-0001.html>

More information about the gdal-dev mailing list