Serving A Large Archive an openpap-client/wms-server

[Gerry Creager]

Frank Warmerdam wrote:
> On 7/8/05, John Bonk <jpbonk at aol.com> wrote:
> 
>>We have a lot of data on an opendap server. We had plans for a
>>MapServer opendap-client/wms-server--in other words we wanted to build
>>a MapServer that will serve raster data from our opendap server.
>>
>>  We were working on a prototype that will serve one of our products.
>>The product consists of monthly files, each file contains 2 parameters
>>per file. At present there are over 80 months (files) of data.
>>
>>  I still consider myself a novice and may have some misconceptions
>>about MapServer, but my understanding is that to offer this data I will
>>need a mapfile with over 160 layers--one for each file times 2 for the
>>2 parameters.
>>
>>  I read in one of the documents
>>(http://mapserver.gis.umn.edu/doc44/mapfile-reference.html) that there
>>is a 50 layer limit for a mapfile. The document said that this limit
>>could be raised by making a change to map.h. However, some of our
>>products have files containing over a hundred parameters. I estimate
>>that if we wanted to offer each of the parameters in each file for
>>several of our products, we would need a mapfile with hundreds of
>>thousands layers. Are we expecting too much from mapserver?
>>
>> Is there any way around this problem?
>>
>>  Does anyone have any suggestions for serving data that is scattered
>>through thousands of files?
> 
> 
> John,
> 
> I think it might make more sense to either produce map objects dynamically
> using mapscript, or create one mapfile for each dataset to serve rather 
> than trying to offer all the products through one master .map file.   

Frank, with some of the datasets we're looking at, your advice is well 
taken, but with some of ours, it's valuable to have a large number of 
'em available for intercomparison.  I don't know what data John's 
talking about but I suspect he's got similar problems to mine.

Wasn't ther a way to increase the number of layers in the source and 
then recompile?  I'm 'bout to get on another plane and don't have time 
to check the code right now, but that's what I recall...

Regards,
Gerry
-- 
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University	
Cell: 979.229.5301 Office: 979.458.4020
FAX:  979.847.8578 Pager:  979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843