[Fwd: Re: [mapserver-users] fastcgi?]

Tue Jun 17 15:59:15 PDT 2003

Hi,

With regard to this thread, I thought I'd give my thoughts on the MapServer
4.o release.

Some of you may or may not be aware of the recent Chameleon software
announced by DM Solutions Group at the MUM
(http://www.maptools.org/chameleon/).  This open source software is the
result of a project (named cwc2) from the Canadian Geospatial Data
Infrastructure (CGDI), part of the GeoConnections program for Canada.  This
software is targeted at enabling our partners and stakeholders to do rapid
application development using standards prescribed by the CGDI and the
OpenGIS Consortium (OGC).

We are currently working with DM Solutions to solidify a Chameleon 1.0 final
release within the next month.  Part of this distribution relies on changes
made to MapServer which is available in the nightly CVS (or what will be the
official 4.0 release).  A MapServer 4.0 is required to satisfy project
requirements.

Having said this, while I wholeheartedly agree with the proposed upgrades to
the software, I ask those who are responsible for official software releases
to consider and provide an estimated release of MapServer 4.0 to the public,
and to consider closing out bugs for 4.0 and to defer new functionality for
a later release.  As MapServer is a separate install in the Chameleon
architecture, we would like to point users to MapServer 4.0, rather than the
nightly build.  

Any comments are appreciated.

Many thanks

..Tom

=========================================
Tom Kralidis
Systems Scientist
GeoConnections / Natural Resources Canada
Tel: (613) 947-1828
http://www.cgdi.ca/
=========================================

> 
> 
> -------- Original Message --------
> Subject: Re: [mapserver-users] fastcgi?
> Date: Mon, 16 Jun 2003 10:47:33 -0400
> From: Frank Warmerdam <warmerdam at pobox.com>
> To: bartvde at xs4all.nl
> CC: "	" 
> <mapserver-users at lists.gis.umn.edu>
> References: <oprqq8eiltmi5aim at localhost>
> 
> Bart van den Eijnden wrote:
>  > Hi list,
>  >
>  > I see that some thread-safety issues have been addressed 
> in the latest
>  > version.
>  >
>  > Are there any plans to implement mapserver as a fast-cgi, 
> so that the
>  > map configuration file does not have to be parsed all the time and
>  > database connections can be "persistent"?
> 
> Bart,
> 
> I discussed this with Daniel a bit on the weekend, and have 
> briefly reviewed
> the FastCGI docs, and I think this would be a useful thing to 
> implement for
> some cases.  I don't think that map file parsing time is likely to be
> significant most of the time, and a straightforward implementation of 
> FastCGI
> support is unlikely to avoid reparsing the mapfile anyways.  
> However, in the
> short term I think it would help a great deal with database 
> connections
> which can be expensive to setup, and initializing the 
> FMEObjects library for
> use with MapServer.
> 
> So, I propose to try and implement simple FastCGI support for 
> MapServer in
> time for the 4.0 release.  This would basically just wrap the 
> mainloop of
> "mapserv".  I will look at enabling multi-threaded fast cgi 
> use, but for
> the usual reasons (big locks around some of the slower 
> sections like GDAL
> reading) running multiple threads is unlikely to help much in 
> some cases.
> 
> What this initial effort will allow for is sharing connection 
> setup, and
> expensive library initializations over a whole bunch of 
> cgi-bin requests
> instead of redoing it for each.  I imagine there will be some 
> sort of a
> --fast-cgi configure option that would build fastcgi support into the
> mapserv executable (or perhaps we will need to link a 
> separate fastmapserv
> executable).  The effort will also involve some checking to 
> ensure there
> aren't per render memory leaks.
> 
> In the future I would like to see:
>    o A mechanism to preserve open handles for GDAL datasets.  
> Some raster
>      formats are a bit expensive to open with GDAL.  Holding open the
>      datasets would also provide dramatic improvments in some 
> cases in that
>      the existing GDAL raster data caching mechanism would 
> then be useful.
>      Note that GDAL already has a concept of "sharing" a GDAL dataset
>      intended to allow multiple parts of an application to 
> share a given
>      GDAL dataset handle via a reference counting mechanism.  
> So I think for
>      this to work well we would just have to implement a 
> mechanism in the
>      .map file (or something else controlling the FastCGI 
> configuration
>      perhaps) to declare GDAL files that should be opened 
> when the server
>      starts and left open till it terminates.
> 
>    o A mechanism to allowing preserving OGR dataset 
> connections similarly.
>      There is also a new "shared dataset" mechanism in OGR so 
> a similiar
>      approach to GDAL should be applicable.  Note that OGR does not 
> generally
>      provide any feature caching logic, so the main savings 
> with OGR caching
>      is to avoid the cost of reopening datasets all the time. 
>  For some
>      formats this is high.
> 
>    o A mechanism to preserve a parsed mapfile.  I don't know 
> if there are
>      issues with this though.  For instance, if URL 
> parameters modify some
>      values in the in-memory mapObj and layerObjs then they 
> can't be reused
>      directly.  Did Sean or someone implement a complete mechanism to
>      copy mapObj's and all the associated stuff?
> 
>    o In the even longer term I would like to implement a more 
> fine grained
>      locking within GDAL so that the "big lock" around GDAL 
> won't be such
>      an issue.
> 
> I don't forsee alot of work implementing caching or preserving access
> to shapefiles.  This is already very lightweight and fast as 
> implemented
> and so should need much work.  Nor would I add will a well organized
> shapefile based map gain much from FastCGI support.  This is mainly
> aimed at various addin libraries.
> 
> I don't know what the implications of FastCGI support for 
> mapscript-cgi
> applications are.  I am not really well organized to deal 
> with that, but
> perhaps once it is working well for mapserv that could be 
> investigated.
> 
> I would add that FastCGI support will also benefit from cleanup in
> MapServ related to multi-threaded support for server loadable modules
> and so forth.
> 
> I am interested in hearing from a few folks who would be interested in
> testing FastCGI support as I am working on it.
> 
> I imagine further FastCGI discussions should be moved to the -dev list
> untill it is working well.
> 
> PS. I really wish I had made it to MUM.  It sounds like it 
> was a blast.
> I have already been talking to the guys about seminars I 
> might present at
> next years meeting on PROJ, GDAL and raster support in 
> general.   Perhaps
> even on FastCGI and multi-threading issues!
> 
> Best regards,
> 
> -- 
> ---------------------------------------+----------------------
> ----------------
> I set the clouds in motion - turn up   | Frank Warmerdam, 
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial 
> Programmer for Rent
> 
> 
> _______________________________________________
> Mapserver-users mailing list
> Mapserver-users at lists.gis.umn.edu
> http://lists.gis.umn.edu/mailman/listinfo/mapserver-users
> 
> 
> -- 
> Dave McIlhagga
> President, DM Solutions Group
> 
> http://www.dmsolutions.ca
> EMail : mcilhagga at dmsolutions.ca
> Phone : 613-565-5056 x15
> Fax : 613-565-0925
> 
> 