[Mapserver-users] Large Map Files

[Steve Lime]

I'm only speculating and have no *real* numbers to base that speculation on. In reality compared to data files it's no biggie to do. It's just that it all adds up. I'm working on using the C pre-processor here and my strategy is to keep each layer in its own file. So a given mapfile may be built from 50 files. I'm also screwing around with defining symbols to turn particular parts of a layer on or off (e.g. query or wms parameters) as needed for a particular application. You can't build that type of logic easily into the mapfile processor, but it's part of the C pre-processor.

I don't mean to put a damper on development moving forward. I think that if enough users want a feature, and it's technically feasible (and there are resources available to do it) then it should be added.

Steve

Stephen Lime
Data & Applications Manager

Minnesota DNR
500 Lafayette Road
St. Paul, MN 55155
651-297-2937
>>> Richard Greenwood <Rich at GreenwoodMap.com> 02/11/03 21:06 PM >>>
I agree with your point that easing management should not be at the expense 
of performance, but how much of a hit do you really think it would be? I 
figure the file system would probably cache the map file(s) because they 
are small and getting read with every hit. The total size it's going to be 
any different (as opposed to rather verbose XML). And if you didn't want to 
use includes you'd just blow them off with no performance penalty.

On the other had, you have mentioned a mapserver specific preprocessor in 
the past, which could potentially answer both issues: convenience with 
includes or XML, and performance by "pre-parseing" the map file. Are you 
thinking of something like make that would be invoked by mapserver each 
time, or a utility that was executed manually? How much performance 
advantage could be achieved by "pre-parseing" the map file?

Rich

At 08:40 PM 2/11/2003 -0600, you wrote:

>Yup, but is that overhead something we really want to  absorb every time a 
>map is created (true whether we're talking map files or xml)? A 
>pre-processor that is run as necessary seems more practical. If using the 
>C pre-processor seems like a kludge then perhaps a tool specific to 
>MapServer needs building.
>
>I worry about the effects of features that ease management headaches at 
>the expense of performance. Which is more important? I agree that 
>segmenting configuration into reusable pieces is very important, but does 
>it have to happen with each rendering of a map?
>
>Steve
>
>
> >>> Paul Ramsey <pramsey at refractions.net> 02/11/03 19:29 PM >>>
>I know nothing about lexers, sadly. However: cannot a relatively
>substitution routine first convert all the INCLUDE's into their
>equivalents, then the full parser runs on the final "file"?
>
>Steve Lime wrote:
> > It's been on the todo list for years. Problem is maintaining parser 
> state over the multiple
> > files. It's possible to have multiple lexers running within the same 
> program but not at the
> > same time. Someone please file this as a feature request...
> >
> >
> >>>>Richard Greenwood <Rich at GreenwoodMap.com> 02/11/03 01:22PM >>>
> >
> > At 11:02 AM 2/11/2003 -0800, Paul Ramsey wrote:
> >
> >
> >>I have been thinking that an extremely powerful extension to the .map file
> >>would be an "INCLUDE" directive, which reads a map file fragment into
> >>another map file.  A quick-and-dirty map service could then be 
> assembled with:
> >
> >
> > An INCLUDE directive would be a wonderful addition!
> >
> > Rich
> >
> >
> > Richard W. Greenwood, PLS
> > Greenwood Mapping, Inc.
> > Rich at GreenwoodMap.com
> > (307) 733-0203
> > http://www.GreenwoodMap.com
> >
> >
> > _______________________________________________
> > Mapserver-users mailing list
> > Mapserver-users at lists.gis.umn.edu
> > http://lists.gis.umn.edu/mailman/listinfo/mapserver-users
>
>
>--
>        __
>       /
>       | Paul Ramsey
>       | Refractions Research
>       | Email: pramsey at refractions.net
>       | Phone: (250) 885-0632
>       \_
>
>_______________________________________________
>Mapserver-users mailing list
>Mapserver-users at lists.gis.umn.edu
>http://lists.gis.umn.edu/mailman/listinfo/mapserver-users
>
>
>_______________________________________________
>Mapserver-users mailing list
>Mapserver-users at lists.gis.umn.edu
>http://lists.gis.umn.edu/mailman/listinfo/mapserver-users


Richard W. Greenwood, PLS
Greenwood Mapping, Inc.
Rich at GreenwoodMap.com
(307) 733-0203
http://www.GreenwoodMap.com