[mapserver-users] best practice: map data structure

[Dan Little]

The most includes I've used in a single file is around 100.  50 or so layers each with a separate label and shape layer.  It worked pretty flawlessly.

There are two solutions to possibly keep in mind:
1 - Shameless Plug) Use an application setup like GeoMOOSE.  We have the application serve many layers in diverse mapfiles to try and simplify the management of mapfiles.  Version 2.0 is even better about this because you can use hybrids of different types of mapfiles to break down the problem however is best for display and performance.

2 - A bit more technical solution) Use a "compiler."  Write a small file format that has a list of the mapfiles you wish to combine together, then write a perl script that combines the files together from the list.  This essentially performs the mapserver includes ahead of time. :-)  I've used this on various projects -- I'm even willing to share source-code ideas if you backchannel me.


----- Original Message ----
> From: Frank Broniewski <liste at geo-dienstleistung.de>
> To: mapserver-users at lists.osgeo.org
> Sent: Monday, June 29, 2009 1:27:34 AM
> Subject: Re: [mapserver-users] best practice: map data structure
> 
> Hi Dan,
> 
> thanks for your response. Using includes for the layers is a nice idea, and it 
> keeps the layer data and configuration together. And you'd get a good overview 
> over all avail. layers on your server just by looking at the existing. 
> directories. Even postgis layers would have a directory since the layers 
> mapfile would be stored there. And you could have several "stylesheet" layer 
> map files in the directory ...
> And in an application directory you could include all the layers necessary for 
> the application in a master mapfile.
> 
> Do you have experience in using lots of includes within a mapfile? I remember 
> using includes for about 30 layers in an php application (pmapper) once (v4.10 
> on ubuntu 7.10) and I run into problems. namely there where too many open 
> files opened by apache and the mapserver process. It seemed, that the includes 
> weren't closed, and each time the map was redrawn, the includes where opened 
> again and so the file system limit of max. open files was reached quite soon.
> 
> On Saturday 27 June 2009 17:08:35 Dan Little wrote:
> > It is very dependent on your needs.  For example, if you have departments
> > responsible for various layers you may want to divide the structure by
> > that, if there is only one administrator then that isn't necessary.
> >
> > Typically, if there is only one administrator I tend to do something like
> > this...
> >
> > [root]/ -- make this whatever you want and that makes sense from a
> > permissions and storage stand point for your new server.
> >
> > for a simple shapefile layer...
> > [root]/layer_a/layer_a.map
> > [root]/layer_a/layer_a.[shp|dbf|shx]
> >
> > for a tree'd shapfile
> > [root]/layer_b/layer_b.map
> > [root]/layer_b/layer_b.[shp|dbf|shx]
> > [root]/layer_b/data/[... quad tree goes here ...]
> >
> > for a raster...
> > [root]/layer_c/layer_c.map
> > [root]/layer_c/raster/[... raster data ...]
> >
> > Then you can use mapserver includes to include the proper mapfile fragments
> > into an applications various mapfiles.
> >
> > ----- Original Message ----
> >
> > > From: Frank Broniewski 
> > > To: mapserver-users at lists.osgeo.org
> > > Sent: Friday, June 26, 2009 8:43:05 AM
> > > Subject: [mapserver-users] best practice: map data structure
> > >
> > > Hello List,
> > >
> > > I want to migrate our old server to a new one, and I want to replace the
> > > somewhat grown map data folder and file mess into something more
> > > structured. Therefore I am looking for some advice on how to organize the
> > > Filesystem Hierarchy. Sure somehow its always a matter of taste and need,
> > > but maybe you have some comments on my proposed structure.
> > >
> > > I browsed already the mailing list archives and the documentation on the
> > > mapserver site but did not find any related infos, but I am willing to
> > > summarize this thread into a howto document if something useful comes up.
> > >
> > > I will be using a linux server, so the structure will reflect the unix
> > > filesystem hierarchy to a certain degree.
> > >
> > > A good starting point seems to be /usr/local/ to me.
> > >
> > > /usr/local/mapdata/ for storing raster and vector files
> > > /usr/local/mapfiles/ for mapfiles and other config files
> > >
> > > You would have /usr/local/mapdata/CLIENTDIR/vector and
> > > /usr/local/mapdata/CLIENTDIR/raster for client specific map data
> > > similar to
> > > /usr/local/mapdata/PROJECTDIR/... for project data
> > >
> > > and /usr/local/mapdata/share/vector & /usr/local/mapdata/share/raster for
> > > shared map data (srtm topo maps e.g.)
> > >
> > > A good structure for mapfiles & co would be:
> > > /mapfile.map
> > > /fonts/ - font config and font files (ttf)
> > > /icons/ - icon files (png, ...)
> > > /symbols/ - symbol.map
> > > /templates/ - template files
> > >
> > > This structure could be mapped to
> > > /usr/local/mapfiles/share/...
> > > or
> > > /usr/local/mapfiles/CLIENTDIR/...
> > >
> > > I would really appreciate any comments and shared experience on this ...
> > >
> > > Listing:
> > >
> > > /usr/local/mapdata/
> > > /usr/local/mapdata/share/
> > > /usr/local/mapdata/share/raster/
> > > /usr/local/mapdata/share/raster/topo20/
> > > /usr/local/mapdata/share/raster/topo20/01.tif
> > > /usr/local/mapdata/share/raster/topo20/...
> > > /usr/local/mapdata/share/raster/topo20/99.tif
> > > /usr/local/mapdata/share/raster/topo20/tileindex.shp
> > > /usr/local/mapdata/share/vector/
> > > /usr/local/mapdata/share/vector/streetnetwork/
> > > /usr/local/mapdata/share/vector/streetnetwork/streets.shp
> > > /usr/local/mapdata/CLIENT/raster/
> > > /usr/local/mapdata/CLIENT/raster/ortho/
> > > /usr/local/mapdata/CLIENT/raster/ortho/01.tif
> > > /usr/local/mapdata/CLIENT/raster/ortho/...
> > > /usr/local/mapdata/CLIENT/raster/ortho/99.tif
> > > /usr/local/mapdata/CLIENT/raster/ortho/tileindex.shp
> > > /usr/local/mapdata/CLIENT/vector/
> > > /usr/local/mapdata/CLIENT/vector/cad/contours.shp
> > > /usr/local/mapdata/PROJECT/raster/
> > > /usr/local/mapdata/PROJECT/raster/climate/
> > > /usr/local/mapdata/PROJECT/raster/climate/precipitation.tif
> > > /usr/local/mapdata/PROJECT/vector/
> > > /usr/local/mapdata/PROJECT/vector/archeology-sites.shp
> > >
> > > /usr/local/mapfiles/
> > > /usr/local/mapfiles/share/
> > > /usr/local/mapfiles/share/mapfile.map
> > > /usr/local/mapfiles/share/fonts/
> > > /usr/local/mapfiles/share/icons/
> > > /usr/local/mapfiles/share/symbols/
> > > /usr/local/mapfiles/share/templates/
> > > /usr/local/mapfiles/CLIENT/
> > > /usr/local/mapfiles/CLIENT/mapfile.map
> > > /usr/local/mapfiles/CLIENT/fonts/
> > > /usr/local/mapfiles/CLIENT/icons/
> > > /usr/local/mapfiles/CLIENT/symbols/
> > > /usr/local/mapfiles/CLIENT/templates/
> > > /usr/local/mapfiles/PROJECT/mapfile.map
> > > /usr/local/mapfiles/PROJECT/fonts/
> > > /usr/local/mapfiles/PROJECT/icons/
> > > /usr/local/mapfiles/PROJECT/symbols/
> > > /usr/local/mapfiles/PROJECT/templates/
> > > _______________________________________________
> > > mapserver-users mailing list
> > > mapserver-users at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 
> 
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users