[Gdal-dev] Wanted: More speed to JPEG2000

[Tom Lynch]

On 11/28/06, Rahkonen Jukka <Jukka.Rahkonen at mmmtike.fi> wrote:
> Hi Tom,,
>
> I wondered if there was something wrong in my compression options and I made an additional test by compressing my test images once more, now with GDAL using JP2ECW driver with default settings.  Now  I think I can continue with my old system because the existing Kakadu set was effectively just as fast as the new JP2ECW set.

Jukka,

I'm not sure exactly what the figures below represent in relation to
the paragraph above, but what I take you to mean is that the
compression scheme you were using, taken from Kakadu recommendations,
gave comparable decoder performance to the JP2ECW default compression
scheme.  I don't find that too surprising given they are definitely
similar (use of RPCL progression order etc.).

I can't comment about the tuning of the JP2KAK driver except to say it
could probably perform a bit better.  The relative performance of our
codecs against Kakadu is usually more comparable, whereas in your
tests you're observing a 3:1 performance ratio in our favour.  Of
course that's not a bad thing from our point of view.  There is also a
possibility that GDAL is better able to exploit our libraries because
of built-in resampling for example (I'm unsure if Kakadu provides
this).

There's more than one factor that could be affecting the variation
between JP2ECW / JP2KAK
- tuning of the compression parameters (maybe not such a big effect)
- GDAL read implementation w.r.t. the interface of the underlying libraries
- implementation of the codecs Kakadu vs. JP2ECW
- platform-specific decoder optimisations (I'm unsure how much
variation this causes for Kakadu)

Finally as you mention the usage pattern represented by your test
probably doesn't reflect a typical MapServer use case.  For example
your testing uses full-res image subsets whereas a typical interactive
map request will not drill down to maximum resolution.

One thing that certainly doesn't surprise me is that ECW is outperforming
JPEG 2000.  The ECW format sacrifices a lot of flexibility for
excellent performance in this type of application and our ECW codec
has always been a bit ahead of the game as compared to our JP2 codec.

> I noticed some change in the general speed of my system from what it used to be so I repeated the test for other image formats as well to be sure I am comparing the same thing.   Here is the summary:
>
> Speed (throughput from server to client hard disk as images/minute)
> GeoTIFF                                  246
> ECW                                        123
> JPEG2000 with JP2ECW         77
> (own parameters)
> JPEG2000 with JP2ECW         73
> (GDAL JP2ECW defaults)
> JPEG2000 with JP2KAK         23
> (own parameters)
> JPEG2000 with JP2KAK         23
> (GDAL JP2ECW defaults)     21
>
> My test measures just the random access to full resolution and does not tell anything about the possible differences in utilising overviews.  I have also noticed that the differences are not at all this big in operations like translating the whole image to other formats. And I am almost sure that JP2KAK driver is not tuned even close to the limits now.

--
Tom Lynch
Development, ER Mapper
Phone:      +61 8 93882900
Fax:        +61 8 93882901
Email:      tom.lynch at ermapper.com
Web:        http://www.ermapper.com
Forums:     http://forum.ermapper.com