[mapserver-users] How best to serve very large (300GB) raster data?

[Xiaodong Zhang]

Hi John,

We've used MapServer to serve satellite images for many years, with each 
of sizes at least 300 GB. Here are what we do:

1. Don't not let database to or any other application such as ArcSDE to 
handle images. We do use Oracle, but it only saves links to images. Let 
MapServer to access images directly.
2. Use tiles for each images

You can check our site http://dngp.umac.org to evaluate if the 
performance can meet your need. Also note, the server where MapServer 
resides uses NFS to access images and our network is only working at 100 
Mbps, performance can certainly be improved if mapserver has a direct 
(say, FC) connection to images.

Please let me know if you have any more questions.

Xiaodong
John Callahan wrote:
>
> I've been using ESRI products for quite a while. To serve a very large 
> raster dataset I have (about 300 GB), or really any raster or vector, 
> I usually just load the data into ArcSDE installed on Oracle and 
> server through ArcIMS. However, I want to move away from ESRI/Oracle 
> and into a more open environment, with all components being open 
> source if possible.
>
> How best to serve a large raster dataset of about 300 GB? My data is 
> in 2010 tiles, each a 3-band TIF of about 135 MB.
>
> >From my initial research, I like what MapServer can do and all of the 
> potential clients (OpenLayers, ka-Map, and many more.) I also really 
> like PostGIS for vectors...and have already started a project with 
> PostGIS points, being maintained by Quantum GIS and mapped through 
> Google Maps Javascript API and some AJAX/PHP code I wrote. My biggest 
> worry is dealing with the very large rasters.
>
> Ideally, a tile server would be nice but I don't want to create the 
> tiles through an ESRI product (like Arc2Earth or HTMLImageMapper.) I 
> really like the possibility of TileCache (http://www.tilecache.org/) 
> working with MapServer. The immediate question is what would be the 
> raster data source to MapServer? Should I create tiles inside each TIF 
> using gdal_translate and then a shapefile tileindex? Maybe go with 
> compressed JPGs instead of TIFs since they usually have much smaller 
> file sizes? I'm not sure where to start.
>
> Thanks for any guidance you can offer.
>
> - John
>
>
>
> ****************************************
> John Callahan
> Geospatial Application Developer
> Delaware Geological Survey
> University of Delaware
> 227 Academy St, Newark DE 19716-7501
> Tel: (302) 831-2833
> Email: john.callahan at udel.edu
> http://www.dgs.udel.edu
> ****************************************
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>