[mapserver-users] XML mapfile?

Wed May 22 13:44:30 PDT 2002

Hi All!

WARNING -- I'm a now-former Lurker with a longwinded set of ideas and
commentary.  If you've thought of this before and it can't be done,
please tell me why...  Here goes!  THANKS for mapserv!

I've read the comments about XML mapfiles, with relish, and I have some
reasons why I think it must be done (not should, must).  

1)  XML is not _that_ slow, if you use a good parser.  Furthermore, you
can (and should) serialize the DOM once and then re-load the serialized
object, as someone mentioned earlier.
2)  It's easy to "manufacture" XML programmatically by user programs
that might automate the production of sites (because there are dozens of
existing libraries that facilitate creation of XML from data).  
3)  It sure would be nice to have a "map on demand" server (perhaps a
web service using XML-RPC or SOAP?) that allows you to fire a
well-formed XML "mapfile" to the server and receive an image or HTML
page in response.  
4)  Although (4) can be done in mapscript, why bother if an XML mapfile
can do it?  You can document what's on the server (or have the web
service provide XML for the "default" DOM tree for mark-up; woooo,
neato) and folks can use ANYTHING to edit / generate the XML.  I can
imagine a client-side app in, e.g., python, that gets the XML, marks up
the DOM tree, makes a server request to retrieve the rendered maps, then
draws additional (locally-stored) data on top.  One could even make a
client-side drawing library that does coordinate conversions and the
like (hey -- how about mapscript?).
5)  EVERYBODY knows how to write and validate XML already.  This one
feature makes me want to switch right now.  The only 'lint' we have for
mapfiles is mapserv itself.  It's a PAIN to edit the mapfile and hit the
browser to check for errors (for one thing, you need a separate
development web server, and that introduces hassles with user
permissions).  If we have an XML mapfile and a proper DTD, a validating
parser will automatically find poorly-formed XML and missing delimiters
and flag them (imagine not having to find the missing 'END' in a
2000-line mapfile).  Furthermore, 'maplint' could take big-time
advantage of the XML for system management.  I envision a python script
that loads the XML mapfile, then traverses the LAYER, CLASS, and LABEL
definitions and reports whether there are missing files, attributes in
the CLASSITEM or LABELITEM specifications that aren't in the DBF files,
etc... 
5a)  Wow!  'maplint' could even load the XML mapfile for, e.g. version
4.0 mapserv and convert it to a version 4.1 XML mapfile!  
5b)  Inheritance.  You can define "new" versions by subclassing the old
versions.  If it's properly implemented in mapserv, a future mapserv 4.0
could read, e.g. 4.1 XML mapfiles and SKIP over the DOM nodes for the
4.1 features that it doesn't support...  Or 4.1 could "skip and warn"
(instead of failing), when an obsoleted 4.0 feature is encountered. 
It's easy, because the code that processes the information in the DOM
tree is independent of the code that reads the DOM tree.  Web browsers
do it, mapserv can too -- the XML reader uses the DTD to read and
validate the DOM tree; what the code does with that DOM tree is up to
the code...  Fatal mapserv errors displayed on client terminals due to
typos in infrequently used (and thus poorly tested) portions of mapfiles
are bad medicine; XML can help to avoid them.
5c)  Most editors are XML-aware and do stuff like syntax highlighting,
auto-inserting tags, and finding missing tags, angle brackets and the
like.  Who wants to maintain a "mapfile-mode" for emacs or vim?
6)  How about using the XML mapfile to populate a DOM tree, then
programmatically (e.g. mapscript python or perl) build an image in a
non-web-based tool.  WOW.  A "poor man's ArcView" is only a few hundred
lines of PyQT or PyGnome/PyGTK code away (or a thousand or so lines of
line noise for Perl/QT or Perl/GTK programmers.  Just kidding. 
Maybe.).  The program could traverse the DOM tree that specifies the map
structure and construct the whole GUI on the fly, creating GUI widgets
that allow layers to be turned on/off, etc.  And the same DOM tree makes
the pictures!  Save the DOM tree to an XML file, and *PRESTO* your view
is on the web!
6a)  Incidentally, one could document the options available for mapserv
objects using another XML file (I'm not an XSL expert, but maybe that's
the way?).  Then the nifty mini-ArcView could build all the layer option
dialog boxes on the fly, radio/checkbox arrays, label options, and the
whole garbanzo...  
7)  At some level, with appropriate editors, XML and DTD's are
self-documenting.  This is a big winner, since one presumes that new
functionality is coming all the time.  Properly designed, nearly all of
the (6) and (6a) code could be independent of mapserv revisions; unless
a new major option needs to be added, the GUI code should require few
revisions to keep up with mapserv.

Although XML is not required to do any of these things, XML facilitates
the incorporation of  mapserver or mapscript or any number of future
children of mapserv/mapscript into other software systems, with
consistent results.  Pardon me, but I'm thinking BIG -- mapserv is a
powerful, flexible, extensible, open, facility for rendering all kinds
of geographic information, FAST.  It can be a component for larger
software systems, with "not-too-much" effort.  If all you're doing is
blowing pictures into web browsers, forget XML.  But for everything
else, one doesn't want to have to write and maintain (a) a mapfile
parser/generator of one's own, or (b) a mapscript application for every
single part of my application.  

I've done some fiddling with "XML mapfiles".  I designed that work as an
input vehicle for python/mapscript.  It is a simple but tedious task to
build a DTD for mapfiles, but I have no control over where mapserv goes,
so I abandoned the effort for now.  If you have an XML mapfile and
python with mapscript, it should be possible (perhaps even simple but
tedious) to implement all of mapserv's functionality in an OO code that
is easily extended (by inheritance) for new applications, either on the
web or standalone.  The resulting classes could be used as a starting
point for all the stuff I outlined above.  I'm way too busy to write it
all myself, but I have a general design in place.  This can probably all
be done outside of the mapserv development effort, but following mapserv
development would be a pain.  It would be nicer to do it all together,
and make a single coherent component infrastructure.

When I imagine a "mapscriptified" OpenEV/ArcView clone with XML mapfiles
as a standard "view" file format, mapserv using the same XML files to
render web pages, python with "XML mapfile" objects plugged into Zope,
and some/most/all of the data in PostgreSQL/PostGIS, well, I could just
cry...  One could even make python mapserv COM objects that plug into
Office, but why go there...

I'll shut up now.  THANKS!
Vic

PS: Count me in on the conference if there is one...

On Wed, 2002-05-22 at 11:04, Puneet Kishor wrote:

    I have wondered about XML myself, and thought, at times, that it would 
    be cool to have XML. Now, reading Daniel's reasoning, I must say what he 
    says makes sense... besides the cool factor, we have to question what 
    XML will do for us. XML facilitates inter-application communication... 
    other than that it is a pain in the ass. If Mapserver is the only 
    program gonna be reading the mapfile, we can make it whatever we want it 
    to, and what it is now is good enough.

    However, in the spirit of discussion, there is one area where XML might 
    be helpful. Most code editors (Homesite, jEdit, Pepper) nowadays can 
    parse XML to create an XML tree. This can make editing a complex Mapfile 
    easier as one can check for node closures, correctness and completeness.

    Other than that, I don't see any great advantage to XML.

    pk/

    On Wednesday, May 22, 2002, at 07:19  AM, Daniel Morissette wrote:

    > C F wrote:
    >>
    >> It seems that this and many other problems I encounter could be solved 
    >> by
    >> making the .map file format XML based.  I searched the archives and 
    >> have
    >> seen quite a bit of discussion on it but I didn't a conclusion.  The 
    >> benefit
    >> for me being that I could download and easily parse and retrieve any
    >> information I needed using XPath.  Is this being considered at all?  
    >> If so,
    >> I'd be happy to contribute to the discussion and development 
    >> (wherever I
    >> could, but I don't know the C language yet) if that would help.  A
    >> combination of an XML mapfile, XSLT templates and XPath extremely 
    >> powerful
    >> compared the current template files.
    >
    >
    > I think that Steve has been considering a switch to XML eventually, but
    > I don't think there is any formal plan.
    >
    > I agree that XML might give more flexibility to the user by allowing him
    > to read the mapfile from external tools, but it would be at the cost of
    > performance (i.e. speed) when parsing the mapfile for every request.
    > Combine this loss of performance with the fact that XML would be harder
    > to edit manually than the current format, and you know where I stand
    > personally.
    >
    > I'm not saying that if we were redesigning from scratch I wouldn't
    > consider XML (we use it a lot in other apps), but since we've already
    > got a very good (fast and simple) mapfile format then I won't be the one
    > initiating a switch to XML mapfiles.
    >
    > Note that MapScript already gives you access to almost every piece of
    > information from the mapfile, so when the CGI cannot do what you need
    > then the logic path is to switch to MapScript, and then sky is the
    > limit.
    >
    > My 0.02$
    > --
    > ------------------------------------------------------------
    >  Daniel Morissette               morissette at dmsolutions.ca
    >  DM Solutions Group              http://www.dmsolutions.ca/
    > ------------------------------------------------------------
    >   Don't put for tomorrow what you can do today, because if
    >       you enjoy it today you can do it again tomorrow.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20020522/fa4cdf84/attachment.htm>