<HTML>
<HEAD>
<TITLE>Re: [UMN_MAPSERVER-USERS] mapserver raster documentation</TITLE>
</HEAD>
<BODY>
<FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'>John -<BR>
<BR>
The only other thing I’d add to the mix is the suggestion that you try to get this data on an external FireWire drive rather than on 2,500 DVDs. We’ve been selling large collections of US GeoTIFF imagery for years and we almost never use DVDs for distribution – and certainly never more than 5 or 6 for a single order. If you’ve got around a terabyte of imagery you can fit that on a single external drive (or come close to it). That is a huge convenience for you in loading the data, and when you’re processing it you can simply connect that drive as your input source and pull data directly from it so you don’t need to worry about running out of source data in a batch job. If you’re acquiring this data from a vendor I would expect they’d prefer to deliver it that way, too.<BR>
<BR>
You will also see better processing performance if your input and output files (at each step) are on different disks so you’re not simultaneously reading from and writing to the same device.<BR>
<BR>
- Ed<BR>
-- <BR>
Ed McNierney<BR>
President and Chief Mapmaker<BR>
Maps a la carte, Inc. / TopoZone.com<BR>
73 Princeton Street, Suite 305<BR>
North Chelmsford, MA 01863<BR>
Phone: (978) 251-4242<BR>
Fax: (978) 251-1396<BR>
ed@topozone.com<BR>
<BR>
<HR ALIGN=CENTER SIZE="3" WIDTH="95%"><B>From: </B>John Mitchell <mitchelljj98@GMAIL.COM><BR>
<B>Reply-To: </B>John Mitchell <mitchelljj98@GMAIL.COM><BR>
<B>Date: </B>Sun, 17 Dec 2006 09:53:44 -0500<BR>
<B>To: </B><MAPSERVER-USERS@LISTS.UMN.EDU><BR>
<B>Subject: </B>Re: [UMN_MAPSERVER-USERS] mapserver raster documentation<BR>
<BR>
Frank,<BR>
<BR>
The reason why I had:<BR>
OUTPUTFORMAT<BR>
NAME png<BR>
DRIVER "GD/PNG"<BR>
MIMETYPE "image/png"<BR>
IMAGEMODE RGB<BR>
EXTENSION "png"<BR>
END<BR>
listed was that for ECW files you had DRIVER "GDAL/JP2ECW" and I am wondering if you need a special driver for Imagine format. <BR>
<BR>
In the next few months I may have to ingest 2500 DVD's of GeoTiff imagery data covering portions of the United States with each DVD contain 1 image of about 1.1 GB in size. In looking over the MapServer Raster documentation I could do the following: <BR>
<BR>
a.) Keep as GeoTiff format<BR>
1.) Convert all the images from whatever their current projection is to WGS84 using GDALWARP.<BR>
2.) Build internal tiling to each GeoTiff image utilizing: gdal_translate -co TILED=YES orig.tif tiled.tif<BR>
3.) Build internal overviews using: gdaladdo -r average *.tif 2 4 8 16 32 64 128<BR>
4.) Build external tiling using: gdal_index doq_index.shp doq/*.tif This will link all the geotiff's in the doq folder to the doq_index.shp, which then is referenced in tileindex within layer instead of DATA. <BR>
5.) ?When using tile indexes to manage many raster files as a single file, it is especially important to have a overview layer that kicks in at high (should'nt this say low instead?) scales to avoid having to open a large number of raster files. Building external overviews use either gdal_merge or gdalwarp (eg gdalwarp -rc -tr 250 250 doq/*.tif overview.tif) The 250 within the gdalwarp statement means a output resolution of 250 meters?<BR>
<BR>
b.) Convert from GeoTiff to Imagine format<BR>
1.) Convert all the images from whatever their current projection is to WGS84 using GDALWARP. <BR>
2.) Mosaic all the GeoTiff's and convert to Imagine format using: gdalwarp -of HFA doq/*.tif tiled.img<BR>
3.) Build internal overviews using: gdaladdo -r average tiled.img 2 4 8 16 32 64 128<BR>
<BR>
Which method would you recommend a or b as far as performance? <BR>
<BR>
Thanks,<BR>
<BR>
John J. Mitchell <BR>
<BR>
<BR>
<BR>
On 12/16/06, <B>Frank Warmerdam</B> <warmerdam@pobox.com> wrote:<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'>John Mitchell wrote:<BR>
> Hi,<BR>
><BR>
> I have been reading over the mapserver raster documentation and I have a<BR>
> few questions:<BR>
><BR>
> 1.) It mentioned a 4 gb limit for GeoTiff files. Is this for GeoTiff <BR>
> files or that GeoTiff's can only be tiled up to 4 gb. Is this 4 gb<BR>
> limit an issue with mapserver or GeoTiff.<BR>
<BR>
John,<BR>
<BR>
The 4GB limit is inherent in TIFF, and it isn't specific to mapserver<BR>
in any way. Note that you can setup a tile index with my TIFF files <BR>
smaller than 4GB. There is no meaningful limit on the amount of data<BR>
that can be in a tileindex.<BR>
<BR>
> 2.) Assuming the 4 gb GeoTiff limit then the GeoTiff would need to be<BR>
> converted to Erdas Imagine (HFA) format. The Documentation said that <BR>
> the Imagine format is already Tiled. Correct?<BR>
<BR>
Imagine format is inherently tiled, normally with 64x64 blocks.<BR>
<BR>
> 3.) Is the format the same to access an Imagine file as it is to access<BR>
> a GeoTiff? I have a sample .map segment that I use to access a GeoTiff: <BR>
<BR>
Yes, Imagine files are accessed the same way in mapserver as GeoTIFF files<BR>
are.<BR>
<BR>
> OUTPUTFORMAT<BR>
> NAME png<BR>
> DRIVER "GD/PNG"<BR>
> MIMETYPE "image/png"<BR>
> IMAGEMODE RGB <BR>
> EXTENSION "png"<BR>
> END<BR>
<BR>
I'm not sure why this OUTPUTFORMAT declaration is in your email. It didn't<BR>
seem to relate to the text of question 3. But I will say that output formats<BR>
is to some degree a distinct issue from input raster formats. <BR>
<BR>
Best regards,<BR>
--<BR>
---------------------------------------+--------------------------------------<BR>
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com <a href="mailto:warmerdam@pobox.com"><mailto:warmerdam@pobox.com></a> <BR>
light and sound - activate the windows | <a href="http://pobox.com/~warmerdam">http://pobox.com/~warmerdam</a><BR>
and watch the world go round - Rush | President OSGeo, <a href="http://osgeo.org">http://osgeo.org</a> <a href="http://osgeo.org"><http://osgeo.org></a> <BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'><BR>
<BR>
<BR>
-- <BR>
John J. Mitchell<BR>
</SPAN></FONT>
</BODY>
</HTML>