Raster TIF format Help

[Jan Hartmann]

Thanks Ed, this is really illuminating. Could you (or someone else) put 
this valuable information somewhere in the MapServer documentation?

Jan

Dr. J. Hartmann
Department of Geography
University of Amsterdam

Ed McNierney wrote:
> Steve -
> 
> I'm not sure who that "al" guy is, but I'll put in my two cents <g>.
> 
> My primary data organizational philosophy is that the optimal MapServer
> image request would result in reading exactly one input TIFF image file,
> and using the entire contents of that image without resampling.
> 
> So - step 1 would be to ensure you have TIFF images, preferably
> uncompressed, and preprocessed with precomputed overviews for all view
> scale settings.  Doing that will get you most of the way there.
> 
> I do not think there is any significant difference between GDAL
> in-the-file overviews created with gdaladdo or separate files created
> using gdalwarp, with each site of files in a LAYER definition limited by
> a MINSCALE/MAXSCALE setting and included in a GROUP.  Using the gdaladdo
> approach produces files that are a little easier to handle (only one
> LAYER definition, no MINSCALE/MAXSCALE fiddling), but it is somewhat
> limiting if you have to cover a wide range of view scales.
> 
> By "somewhat limiting" I'm referring to the fact that if you support
> view scales that are much lower resolution than the original input
> files, you can end up (in a zoomed-out situation) needing to read MANY
> input files in order to produce one MapServer image, thus drifting a
> long way from the optimal arrangement I mentioned above.  This can be a
> noticeable performance drag.  If you're using the "external overviews"
> approach (via gdalwarp, not internal overviews) then you will have the
> option of mosaicing the now-tiny files together to produce larger
> images.  This limits your ability to do some shortcuts with TILEINDEX
> creation, as you will now have a necessarily distinct set of images that
> will need their own index.
> 
> I have not found there to be much benefit from internal TIFF tiling
> (using TILED=YES in GDAL creation options), but it certainly can make
> some difference if your file width is much larger than the size of the
> output image.  That's because you will need to read less data from the
> TIFF file in order to get the contents of the area you need.  However,
> sequential data reads from a file are fast and are very likely to be
> "free" in that your disk is going to read several sequential sectors all
> at once whether you like it or not!  I don't think it hurts, so I'd go
> ahead and adopt it as a general approach.
> 
> I believe, however, that internal tiling would be more valuable for
> compressed TIFFs, especially LZW-compressed images, due to the need to
> read from the beginning in order to do the decompression.  LZW
> compression is of little benefit for most imagery and I don't recommend
> it.  LZW compression can be appropriate for your topo map images if they
> are relatively clean scans.  On the other hand, they tend not to be very
> large data sets compared to the imagery so you may not be motivated to
> compress them.
> 
> So - keep it simple.  Use uncompressed TIFFs precomputed at all
> available output resolutions.  Use TILED=YES when creating TIFFs.  Use
> gdaladdo internal overviews if the convenience is helpful, but don't
> create lots of tiny TIFFs at zoomed-out scales; use gdalwarp and then
> mosaic the images together.  Try to produce input images that are, in
> general, no smaller than the output image sizes you support.
> 
> GDAL does not support as many resampling options as other applications
> do, and I have preferred roll-my-own bicubic sampling for zoomed-out
> overviews, so the "external overviews" approach can be preferable.  But
> it does require more data and mapfile maintenance work.
> 
> I hope this helps; I'm being a bit telegraphic as I know you've handled
> a few rasters before!  Let me know if anything is especially cryptic.
> 
>      - Ed
> 
> Ed McNierney
> TopoZone.com
> 
> -----Original Message-----
> From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On
> Behalf Of Stephen Woodbridge
> Sent: Sunday, March 04, 2007 12:07 PM
> To: MAPSERVER-USERS at LISTS.UMN.EDU
> Subject: [UMN_MAPSERVER-USERS] Raster TIF format Help
> 
> Ed McNierney, et al,
> 
> What is the conventional knowledge on how to organize TIF images 
> internally for optimal access by mapserver?
> 
> I remember that TIF files can have a lot of different internal 
> organizations and that some are much more efficient for mapserver to 
> access when serving data.
> 
> I am looking at putting together some topo map scans and Landsat imagery
> 
> of the middle east. The topo maps are roughly 4000x3000 pixels and in 
> various scales like 500k, 200k, 100k, 50k, and the landsat imagery is 
> looks to be in 1 deg squares.
> 
> Can GDAL be used to reorganize the images and if so what options should 
> I specify? or are there other tools I need to use?
> 
> Thanks,
>    -Steve
>