VS: [Gdal-dev] Wanted: More speed to JPEG2000

[Rahkonen Jukka]

> Lähettäjä: Tom Lynch 

> On 11/28/06, Rahkonen Jukka <Jukka.Rahkonen at mmmtike.fi> wrote:
> > Hi Tom,,
> >
> > I wondered if there was something wrong in my compression 
> options and I made an additional test by compressing my test 
> images once more, now with GDAL using JP2ECW driver with 
> default settings.  Now  I think I can continue with my old 
> system because the existing Kakadu set was effectively just 
> as fast as the new JP2ECW set.
> 
> Jukka,
> 
> I'm not sure exactly what the figures below represent in relation to
> the paragraph above, but what I take you to mean is that the
> compression scheme you were using, taken from Kakadu recommendations,
> gave comparable decoder performance to the JP2ECW default compression
> scheme.  I don't find that too surprising given they are definitely
> similar (use of RPCL progression order etc.).
> 
> I can't comment about the tuning of the JP2KAK driver except to say it
> could probably perform a bit better.  The relative performance of our
> codecs against Kakadu is usually more comparable, whereas in your
> tests you're observing a 3:1 performance ratio in our favour.  Of
> course that's not a bad thing from our point of view.  There is also a
> possibility that GDAL is better able to exploit our libraries because
> of built-in resampling for example (I'm unsure if Kakadu provides
> this).
> 
> There's more than one factor that could be affecting the variation
> between JP2ECW / JP2KAK
> - tuning of the compression parameters (maybe not such a big effect)
> - GDAL read implementation w.r.t. the interface of the 
> underlying libraries
> - implementation of the codecs Kakadu vs. ECW
> - platform-specific decoder optimisations (I'm unsure how much
> variation this causes for Kakadu)
> 
> Finally as you mention the usage pattern represented by your test
> probably doesn't reflect a typical MapServer use case.  For example
> your testing uses full-res image subsets whereas a typical interactive
> map request will not drill down to maximum resolution.
> 
> The one thing that doesn't surprise me is that ECW is outperforming
> JPEG 2000.  The ECW format sacrifices a lot of flexibility for
> excellent performance in this type of application and our ECW codec
> has always been a bit ahead of the game as compared to our JP2 codec.


Tom,

You had got the main information out of my figures even they included a typo. Different packages are making similarly behaving JPEG2000 images when the same compression instructions are used (not really a big surprise) and the combination of GDAL and JP2KAK driver is no good in reading full resolution subsets.

I think I am now quite satisfied with the information about today's situation with using ECW and JPEG2000 through GDAL and MapServer. My interpretation for our own environment is that with Mapserver around 50% of the speed can be achieved with 10-20% of disk space by using compressed,  high quality ECW images. Using JPEG2000 compression is throttling the Mapserver output to 30% of the optimal but unlike ECW it gives a possibility to use lossless compression with some 50% savings in disk space (comparison between uncompressed TIFF with overviews and lossless JPEG2000).  I think that the formala is not very favourable for JPEG2000.  That's sad because it will mean having our imagery in at least two different formats, JPEG2000 for archive and TIFF for daily use, or possibly ECW for not-so-often-requested layers.


Regards,

-Jukka-

 
> > I noticed some change in the general speed of my system 
> from what it used to be so I repeated the test for other 
> image formats as well to be sure I am comparing the same 
> thing.   Here is the summary:
> >
> > Speed (throughput from server to client hard disk as images/minute)
> > GeoTIFF                                  246
> > ECW                                        123
> > JPEG2000 with JP2ECW         77
> > (own parameters)
> > JPEG2000 with JP2ECW         73
> > (GDAL JP2ECW defaults)
> > JPEG2000 with JP2KAK         23
> > (own parameters)
> > JPEG2000 with JP2KAK         23  <<<<<this number should not exist here!
> > (GDAL JP2ECW defaults)     21
> >
> > My test measures just the random access to full resolution 
> and does not tell anything about the possible differences in 
> utilising overviews.  I have also noticed that the 
> differences are not at all this big in operations like 
> translating the whole image to other formats. And I am almost 
> sure that JP2KAK driver is not tuned even close to the limits now.
> 
> -- 
> Tom Lynch
> Development, ER Mapper
> Phone:      +61 8 93882900
> Fax:        +61 8 93882901
> Email:      tom.lynch at ermapper.com
> Web:        http://www.ermapper.com
> Forums:     http://forum.ermapper.com
>