Map file with 35,000+ Lines... management issue

[Bob Basques]

Ed,

While I do have 72 STYLE def's, I only have 11 layers defined in my Googlish mapfile, that's 11 layer definitions for essentially the same data type, Street centerlines.  That would equate to approximately 60 real layers being queried, granted, the googlish mapfile I've been working on is somewhat complex, but I would venture that most layers could be as complex with similar cartographic needs.

A couple of the layers i have are for Legends and such for additional output control, and if the control were not a big factor, they could be removed altogether if needed.   But it does point to the need in some circles (like those of us who fancy ourselves as some sort of fancy "Cartographer" ;c) to be able to add in our flourishes to the outputs that do indeed require defining more than one layer.

Some instances will require these mapfiles to get very large, and the standards bar is being risen very fast IMHO, and is getting harder and harder to keep up with some of the average user expectations for readability as well as printabilty of online maps.  This is a high priority item with me, mostly because I don't want to be stuck doing cartography day in and day out.   I'm hoping that there will be some discussion to come out of this Thread related to developing some different methods for applying styles that may end up being easier to maintain as well as easier to teach to others.

bobb





Bob Basques
GIS Systems Developer
City of Saint Paul, MN

GISmo 
Powered by
GeoMOOSE




>>> Ed McNierney <ed at TOPOZONE.COM> wrote:

Ritesh -

Obviously you cannot possibly need 658 layers to respond to a single map =
request.  The first thing to do is figure out how to split up this one =
map file into multiple map files and then use some front-end logic to =
select the appropriate map file to serve a particular request.  You may =
find the INCLUDE directive helpful, but you should simply begin by =
eliminating the unnecessary.  The theoretical ideal would be that a =
single MapServer request would specify a map file that contained nothing =
more than the symbols, fonts, and layers required to serve that single =
request - nothing more.

- Ed

Ed McNierney
Chief Mapmaker
Demand Media / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA=A0 01863
ed at topozone.com
Phone: +1 (978) 251-4242
Fax: +1 (978) 251-1396

-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On =
Behalf Of ritesh ambastha
Sent: Wednesday, January 23, 2008 2:44 PM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] Map file with 35,000+ Lines... =
management issue

Thanks Bob,

The map file consists of 658 Layers.
It runs with openlayers and postgis.

Now, am trying to sort out the best way for solving this issue.
Your reply helped me to view at the problems+solutions in broad =
spectrum..

Warm Regards,
Ritesh

On Jan 24, 2008 1:04 AM, Bob Basques <Bob.Basques at ci.stpaul.mn.us> =
wrote:
>
>
> Ritz,
>
> Whew, 35k lines, that big.  How many layers is that anyway?   The =
Googlish
> mapfile I just did only has 1100 lines in it, and that's mostly for
> readability.  I could probably get it down to half that size if I =
tried.
>
> Don't know what I can contribute as a "Best Practice", because I feel =
that
> in most cases, that form follows function, if you need a capability, =
you
> build it.  Anyway, here are some of my thoughts.
>
> These same sorts of performance questions crossed my mind too.  The =
Googlish
> mapfile I've been working on has 72 separate STYLE definitions for =
example.
> Mostly ranged around threshholding of certain styles.  I can see that =
adding
> in the Water bodies, Railroad, Parks, and such, is really going to =
make this
> thing big.  I may just do those as separate MapFiles though since =
GeoMoose
> handles things like these separations very nicely.
>
> These are some of the primary reasons that contributed to the way =
we've
> built GeoMoose as a client and why it runs against MapServer CGI, so =
that it
> can abstract the layer calls in this fashion.  We're running 135+ =
layers
> internally at the moment, and they all have their own MAPFILE and are =
all
> called separately from the client.  It has made life much easier with =
regard
> to MapFile creation and maintenance, since each data custodian handles =
their
> respective MAPFILE.  The performance issues are minimized well since =
even
> the data intensive layers are not too bad from a performance =
standpoint.
>
> But even my Googlish looking mapfile got prettty big (in my opinion) =
for
> simply displaying centerlines of streets.   I've learned quite a bit =
from
> these exercises about these types of questions.  While I have yet to =
attack
> the performance side of things, I anticpate that I'll need to =
segregate the
> data out at differing thresholds in order to gain some performance =
boots.
> We're all about doing dynamic requests here since many of our datasets
> change very frequently, in some cases, down to the minute.  I may look =
into
> tiling at some point in the future, but it will still be only for some =
of
> the layers, there will still be a need to have this dynamic request
> structure in place.
>
> bobb
>
>
>
>
>
>
> Bob Basques
> GIS Systems Developer
> City of Saint Paul, MN
>
>
> GISmo
> Powered by
> GeoMOOSE
>
>
> >>> riteshambastha <ritesh.linux at GMAIL.COM> wrote:
>
> Dear Readers,
>
> I have developed a map file with more than 35,000 Lines. Its size will =
grow
> by double/triple in next few months. Now, I am trying to tune my map =
file by
> removing unwanted lines. Still, I am bit confused about its =
maintenance.
>
>
> Please throw some lights over writing map files by following best =
practices.
>
> Thanks in advance.
>
> Regards,
> Ritz
> --
> View this message in context:
> =
http://www.nabble.com/Map-file-with-35,000+-Lines...-management-issue-tp1=
5048892p15048892.html
>
> Sent from the Mapserver - User mailing list archive at Nabble.com.
>



--=20
Ritesh Ambastha,

Project Manager
Mobiance Technologies,
Bangalore

+91-80-41264755
-------------- next part --------------
Skipped content of type multipart/related