[OSGeo-Discuss] Open File Formats andProprietaryAlgorithms[SEC=UNCLASSIFIED]

[Christopher Schmidt]

On Fri, Aug 21, 2009 at 09:45:04AM -0700, Landon Blake wrote:
> MPG wrote: "Tiling" essentially means you can take a large file and
> compress pieces of it independently.  This avoids having to deal with
> the large memory footprint issues, but it can also lead to seam-line
> artifacts under certain conditions.  Ideally, one would prefer to have
> the option of compressing large images without resorting to using
> tiles."
> 
> This is probably a stupid question, since I know absolutely nothing
> about image compression, but couldn't you overlap the tiles slightly to
> avoid the seam lines?
> 
> This would obviously result in a slightly larger file size because some
> pixels would be compressed twice. But that might be OK if you were
> trying to compress a huge image.
> 
> What about reading chunks of the image off disk, instead of trying to
> put the whole image in memory? This would be slower, but might make an
> impossible task possible.

"Reading chunks of image off disk" == "tiling". With compression, bits
aren't stored on th disk in a way that you can say "Okay, bytes 0-32768
are the first 720 pixels" in any way. Instead, you have to decompress
the image, or part of it, to start to learn these things. Tiling lets
you split the image up into many little chunks, which you can read
individually.

> We run into this problem with vector datasets to. Some datasets are just
> to stinking BIG. One of my tasks for OpenJUMP is to write a core module
> that displays vector data accessed directly from disk, instead of from
> memory. This will be slower, but it is better than crashing the program
> because there isn't enough RAM.

Most Vector datasets have some lvel of "random access" -- I can look for
feature 7, and get it, because i know where the start and end of feature
7 are. I don't know where the start and end of pixel 7 is -- because
its' different depending on exactly how wth file is compressed.

This is all a vast simplification, and some of it is probably
not-entirely right, but the problems are -- as you suggested -- more
complex than most people not working in imagery know. (And even more
complex than some of them know, most likely :))

Regards,
-- 
Christopher Schmidt
Web Developer