[OSGeo-Discuss] 'lossless' JPEG2000

[Ed McNierney]

Christopher -

Let me add the "evidence" that I have found that reducing the strip size
in LZW-compressed GeoTIFFs has, not surprisingly, a VERY large effect on
read performance - about a factor of 10 in the particular cases I used.
That indicates that the data you report might not be a good 'order of
magnitude' measure of performance.  I'm not talking about "subtler"
effects of memory, caching, etc. but the substantial effect of changing
the LZW strip size in a large image; the amount of work required to
decompress a portion of that image is very, very different from the case
in which the entire file is treated as one strip, due to the
dictionary-building nature of the LZW algorithm.

You're welcome to draw conclusions based on one data point; I am
concerned that readers of your post may unreasonably extrapolate from
them.  You're welcome to stick to them, but I'm not sure I'd recommend
that other users stick to them, at least in situations outside of your
test environment.

     - Ed

-----Original Message-----
From: discuss-bounces at lists.osgeo.org
[mailto:discuss-bounces at lists.osgeo.org] On Behalf Of Christopher
Schmidt
Sent: Monday, February 25, 2008 9:18 PM
To: discuss at lists.osgeo.org
Subject: Re: [OSGeo-Discuss] 'lossless' JPEG2000

On Mon, Feb 25, 2008 at 09:04:59PM -0500, Ed McNierney wrote:
> Christopher -
> 
> You will very likely find that using different LZW compression options
(particularly setting a small strip size) will slightly degrade
compression performance while significantly improving read time.  While
I think your test data are valid, they only address one of many possible
configurations and I wouldn't necessarily make broad generalizations
about LZW from them.
> 
> However, I have generally found that LZW compression for photographic
data is indeed not a good choice; I'm surprised you got it to perform as
well as you did (in compression).

Yeah, I think we've stumbled back and forth across these numbers before.
I'm aware that they're essentially 'back of the envelope': they weren't
run entirely in isolation, they were only repeated a couple of times
(half dozen rather than an order of magnitude more), they might have
been cached in memory, etc. etc. etc. However, they do seem to serve as
a good 'order of magnitude' measure of size and performance for
compressing aerial imagery based on other similar experiments, and I
have no evidence to seriously discount them, so I'm sticking to them.

Regards,
-- 
Christopher Schmidt
Web Developer
_______________________________________________
Discuss mailing list
Discuss at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss