mapserver raster documentation

Mon Dec 18 09:43:27 EST 2006

John -

The only other thing I¹d add to the mix is the suggestion that you try to
get this data on an external FireWire drive rather than on 2,500 DVDs.
We¹ve been selling large collections of US GeoTIFF imagery for years and we
almost never use DVDs for distribution  and certainly never more than 5 or
6 for a single order.  If you¹ve got around a terabyte of imagery you can
fit that on a single external drive (or come close to it).  That is a huge
convenience for you in loading the data, and when you¹re processing it you
can simply connect that drive as your input source and pull data directly
from it so you don¹t need to worry about running out of source data in a
batch job.  If you¹re acquiring this data from a vendor I would expect
they¹d prefer to deliver it that way, too.

You will also see better processing performance if your input and output
files (at each step) are on different disks so you¹re not simultaneously
reading from and writing to the same device.

     - Ed
-- 
Ed McNierney
President and Chief Mapmaker
Maps a la carte, Inc. / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
Phone: (978) 251-4242
Fax: (978) 251-1396
ed at topozone.com

From: John Mitchell <mitchelljj98 at GMAIL.COM>
Reply-To: John Mitchell <mitchelljj98 at GMAIL.COM>
Date: Sun, 17 Dec 2006 09:53:44 -0500
To: <MAPSERVER-USERS at LISTS.UMN.EDU>
Subject: Re: [UMN_MAPSERVER-USERS] mapserver raster documentation

Frank,

The reason why I had:
OUTPUTFORMAT
   NAME png
   DRIVER "GD/PNG"
   MIMETYPE "image/png"
   IMAGEMODE RGB
   EXTENSION "png"
  END
listed was that for ECW files you had DRIVER "GDAL/JP2ECW" and I am
wondering if you need a special driver for Imagine format.

In the next few months I may have to ingest 2500 DVD's of GeoTiff imagery
data covering portions of the United States with each DVD contain 1 image of
about 1.1 GB in size.  In looking over the MapServer Raster documentation I
could do the following:

a.) Keep as GeoTiff format
     1.) Convert all the images from whatever their current projection is to
WGS84 using GDALWARP.
     2.) Build internal tiling to each GeoTiff image utilizing:
gdal_translate -co TILED=YES orig.tif tiled.tif
     3.) Build internal overviews using: gdaladdo -r average *.tif 2 4 8 16
32 64 128
     4.) Build external tiling using: gdal_index doq_index.shp doq/*.tif
This will link all the geotiff's in the doq folder to the doq_index.shp,
which then is referenced in tileindex within layer instead of DATA.
     5.) ?When using tile indexes to manage many raster files as a single
file, it is especially important to have a overview layer that kicks in at
high (should'nt this say low instead?) scales to avoid having to open a
large number of raster files.  Building external overviews use either
gdal_merge or gdalwarp (eg gdalwarp -rc -tr 250 250 doq/*.tif overview.tif)
The 250 within the gdalwarp statement means a output resolution of 250
meters?

b.) Convert from GeoTiff to Imagine format
     1.) Convert all the images from whatever their current projection is to
WGS84 using GDALWARP.
     2.) Mosaic all the GeoTiff's and convert to Imagine format using:
gdalwarp -of HFA doq/*.tif tiled.img
     3.) Build internal overviews using: gdaladdo -r average tiled.img 2 4 8
16 32 64 128

Which method would you recommend a or b as far as performance?

Thanks,

John J. Mitchell 

On 12/16/06, Frank Warmerdam <warmerdam at pobox.com> wrote:
> John Mitchell wrote:
>> > Hi,
>> >
>> > I have been reading over the mapserver raster documentation and I have a
>> > few questions:
>> >
>> > 1.) It mentioned a 4 gb limit for GeoTiff files.  Is this for GeoTiff
>> > files or that GeoTiff's can only be tiled up to 4 gb.  Is this 4 gb
>> > limit an issue with mapserver or GeoTiff.
> 
> John,
> 
> The 4GB limit is inherent in TIFF, and it isn't specific to mapserver
> in any way.  Note that you can setup a tile index with my TIFF files
> smaller than 4GB.  There is no meaningful limit on the amount of data
> that can be in a tileindex.
> 
>> > 2.) Assuming the 4 gb GeoTiff limit then the GeoTiff would need to be
>> > converted to Erdas Imagine (HFA) format.  The Documentation said that
>> > the Imagine format is already Tiled.  Correct?
> 
> Imagine format is inherently tiled, normally with 64x64 blocks.
> 
>> > 3.) Is the format the same to access an Imagine file as it is to access
>> > a GeoTiff?  I have a sample .map segment that I use to access a GeoTiff:
> 
> Yes, Imagine files are accessed the same way in mapserver as GeoTIFF files
> are.
> 
>> >  OUTPUTFORMAT
>> >   NAME png
>> >   DRIVER "GD/PNG"
>> >   MIMETYPE "image/png"
>> >   IMAGEMODE RGB
>> >   EXTENSION "png"
>> >  END
> 
> I'm not sure why this OUTPUTFORMAT declaration is in your email.  It didn't
> seem to relate to the text of question 3.  But I will say that output formats
> is to some degree a distinct issue from input raster formats.
> 
> Best regards,
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
> <mailto:warmerdam at pobox.com>
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | President OSGeo, http://osgeo.org
> <http://osgeo.org>
> 

-- 
John J. Mitchell

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapserver-users/attachments/20061218/80f4f486/attachment.html