Map file with 35,000+ Lines... management issue

[Ed McNierney]

Ritesh -

Obviously you cannot possibly need 658 layers to respond to a single map request.  The first thing to do is figure out how to split up this one map file into multiple map files and then use some front-end logic to select the appropriate map file to serve a particular request.  You may find the INCLUDE directive helpful, but you should simply begin by eliminating the unnecessary.  The theoretical ideal would be that a single MapServer request would specify a map file that contained nothing more than the symbols, fonts, and layers required to serve that single request - nothing more.

	- Ed

Ed McNierney
Chief Mapmaker
Demand Media / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
ed at topozone.com
Phone: +1 (978) 251-4242
Fax: +1 (978) 251-1396

-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of ritesh ambastha
Sent: Wednesday, January 23, 2008 2:44 PM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] Map file with 35,000+ Lines... management issue

Thanks Bob,

The map file consists of 658 Layers.
It runs with openlayers and postgis.

Now, am trying to sort out the best way for solving this issue.
Your reply helped me to view at the problems+solutions in broad spectrum..

Warm Regards,
Ritesh

On Jan 24, 2008 1:04 AM, Bob Basques <Bob.Basques at ci.stpaul.mn.us> wrote:
>
>
> Ritz,
>
> Whew, 35k lines, that big.  How many layers is that anyway?   The Googlish
> mapfile I just did only has 1100 lines in it, and that's mostly for
> readability.  I could probably get it down to half that size if I tried.
>
> Don't know what I can contribute as a "Best Practice", because I feel that
> in most cases, that form follows function, if you need a capability, you
> build it.  Anyway, here are some of my thoughts.
>
> These same sorts of performance questions crossed my mind too.  The Googlish
> mapfile I've been working on has 72 separate STYLE definitions for example.
> Mostly ranged around threshholding of certain styles.  I can see that adding
> in the Water bodies, Railroad, Parks, and such, is really going to make this
> thing big.  I may just do those as separate MapFiles though since GeoMoose
> handles things like these separations very nicely.
>
> These are some of the primary reasons that contributed to the way we've
> built GeoMoose as a client and why it runs against MapServer CGI, so that it
> can abstract the layer calls in this fashion.  We're running 135+ layers
> internally at the moment, and they all have their own MAPFILE and are all
> called separately from the client.  It has made life much easier with regard
> to MapFile creation and maintenance, since each data custodian handles their
> respective MAPFILE.  The performance issues are minimized well since even
> the data intensive layers are not too bad from a performance standpoint.
>
> But even my Googlish looking mapfile got prettty big (in my opinion) for
> simply displaying centerlines of streets.   I've learned quite a bit from
> these exercises about these types of questions.  While I have yet to attack
> the performance side of things, I anticpate that I'll need to segregate the
> data out at differing thresholds in order to gain some performance boots.
> We're all about doing dynamic requests here since many of our datasets
> change very frequently, in some cases, down to the minute.  I may look into
> tiling at some point in the future, but it will still be only for some of
> the layers, there will still be a need to have this dynamic request
> structure in place.
>
> bobb
>
>
>
>
>
>
> Bob Basques
> GIS Systems Developer
> City of Saint Paul, MN
>
>
> GISmo
> Powered by
> GeoMOOSE
>
>
> >>> riteshambastha <ritesh.linux at GMAIL.COM> wrote:
>
> Dear Readers,
>
> I have developed a map file with more than 35,000 Lines. Its size will grow
> by double/triple in next few months. Now, I am trying to tune my map file by
> removing unwanted lines. Still, I am bit confused about its maintenance.
>
>
> Please throw some lights over writing map files by following best practices.
>
> Thanks in advance.
>
> Regards,
> Ritz
> --
> View this message in context:
> http://www.nabble.com/Map-file-with-35,000+-Lines...-management-issue-tp15048892p15048892.html
>
> Sent from the Mapserver - User mailing list archive at Nabble.com.
>



-- 
Ritesh Ambastha,

Project Manager
Mobiance Technologies,
Bangalore

+91-80-41264755