Suggestions for Data-Serving & UI

[Frank Warmerdam]

On 5/11/05, Shoaib Burq <saburq at gmail.com> wrote:
> Do you think mapserver or other OSS technologies today are up to the
> task? If so are there any demos out there? If not what's missing?
> 
> I'll keep this short and provide more info as needed.

Shoaib, 

I do think you can manage your study area as a single file per data
layer using available free software.  My suggestion would be that
you use Imagine format (HFA driver in GDAL) for your datasets.  This
is a tiled format that supports various pixel types, and handles large
files gracefully.  GeoTIFF will be somewhat inappropriate for this
use since it supports datasets only up to 4GB. 

I'm not sure how you are currently rectifying the strips of data you get.
The gdalwarp utility can handle reprojection or polynomial warping
based on control points, and could gracefully warp directly into your
large working image for updates. 

As long as you build overviews (which you would need to refresh after
any updates to the base image) with gdaladdo, the resulting big imagine
file should work fine in MapServer with good performance.   

For preparing custom data extracts, you will likely need to do a bit of 
customization.  I imagine one of the existing JavaScript client setups can
be modified to allow dragging out a request rectangle to send back to the
server for extracts.  It may be difficult to use mapserver to do the actual 
extracts if you want people to be able to pull out very large regions, since
MapServer assembles the whole image to be returned in RAM.  Instead it
might be more suitable to have a custom cgi script that would invoke 
gdal_translate or gdalwarp to extract a selected subregion at a desired
resolution and output format.  

All, in all, it is a doable arrangement though it will require some custom
scripting work (server and client side).   I would add that working with
100GB files can be a bit tedious - though potentially less tedious than 
many medium sized files. 

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent