<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;" dir="ltr">
<p>Thanks Even for updating the doc and also running some perf testing using those datasets.</p>
<p><br>
</p>
<p>The results you get when enabling multi-threading are very interesting: 26.s to 9.9 s. Then using the work in progress you end up with 1s. Basically a 26 times speed improvement.</p>
<p><br>
</p>
<p>Is there a timeline for releasing next version of openJpeg (either master or work in progress).</p>
<p><br>
</p>
<p>I'm very interested in the performance improvement since they looks promising and may get closer or beat to Kakadu performance.</p>
<p><br>
</p>
<p>Are there shared jp2 dataset you're using for performance benchmark ?</p>
<p><br>
</p>
<p>Noureddine</p>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Even Rouault <even.rouault@spatialys.com><br>
<b>Sent:</b> Monday, August 28, 2017 2:55 PM<br>
<b>To:</b> N. Farah<br>
<b>Cc:</b> gdal-dev@lists.osgeo.org<br>
<b>Subject:</b> Re: [gdal-dev] Fwd: [OpenJPEG] OpenJPEG 2.2.0 is out ! Faster and safer.</font>
<div> </div>
</div>
<div>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">On lundi 28 août 2017 18:00:03 CEST N. Farah wrote:</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> Thanks Even for the quick response. I'll give it a try with setting</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> OPJ_NUM_THREADS env variable to 8. Does it need to be used with</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> GDAL_NUM_THREADS ? Any url to read about those two env variables ?</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">I've added some documentation per</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">https://trac.osgeo.org/gdal/changeset/39954</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> The two dataset i used:</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> <http://data.opengee.org/FusionTutorial-Full.tar.gz></p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> https://1drv.ms/u/s!AkFDoDHsFe7_hkBS6GbG4xMqnk6t<http://data.opengee.org/Fus</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> ionTutorial-Full.tar.gz></p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> https://1drv.ms/u/s!AkFDoDHsFe7_hkG1-XDo7UztsPoC<http://data.opengee.org/Fu</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> sionTutorial-Full.tar.gz></p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">Those images are single-tiled, so the GDAL_NUM_THREADS optimization will not help for them, and current openjpeg versions are super inefficient on those use cases, as GDAL expose a pseudo tiling
to avoid a super big tile size, which cause the whole image to be decoded multiple times. I'm currently working on improving subtile decoding, that will substantially improve performance for such images (first steps are already in openjpeg master, next to
follow)</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">I didn't have the patience to convert the whole image, so just took 3 GDAL blocks of 1024x1024</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">Without threading support, on hot runs:</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">gdal_translate bluemarble_4km.jp2 out.tif -srcwin 0 0 3072 1024</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">35.1 s GDAL 2.2 / openjpeg 2.1.2</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">26.2 s GDAL 2.2 / openjpeg 2.2.0</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">So I do see an improvement. Not sure why you don't.</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">Hardware: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">OS: Ubuntu 16.04, 64bit build</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">and as a teaser:</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">3.1 s GDAL 2.2 / openjpeg master</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">1.5 s GDAL 2.2 / openjpeg (my work-in-progress branch)</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">With 8 threads:</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">9.9 s GDAL 2.2 / openjpeg 2.2.0</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">2.3 s GDAL 2.2 / openjpeg master</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">0.9 s GDAL 2.2 / openjpeg (my work-in-progress branch)</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">With usgsLanSat.jp2</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">gdal_translate usgsLanSat.jp2 out.tif -srcwin 0 0 3072 1024</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">40.2 s GDAL 2.2 / openjpeg 2.1.2</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">32.3 s GDAL 2.2 / openjpeg 2.2.0</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">> p.s: i'm using GDAL 2.1.2</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">GDAL 2.1 or 2.2 should have comparable performances on this, I think</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;"> </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">-- </p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">Spatialys - Geospatial professional services</p>
<p style="margin: 0px; text-indent: 0px; white-space: pre-wrap;">http://www.spatialys.com</p>
</div>
</div>
</div>
</body>
</html>