<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7651.59">
<TITLE>MapFile Parsing into DOM, MapFile Schema, and MapFile multilingual support</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P><FONT SIZE=2 FACE="Arial">Hello!</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Good to see many of you at FOSS4G!</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">As discussed with Steve & Hobu and others, I wanted to put in writing my thoughts on this concept I have of parsing MapFiles into a DOM, making it look like XML without it actually BEING XML.</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Background:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">- I was writing, just for fun, a MapFile editor (In Python/wxPython). I ended up settling on the idea that simply treating the MapFile as a structured document (XML-style) was the best and simplest way for me to go. The mapfile being an already fairly structured document, this shouldn't be very hard. And it wasn't.</FONT></P>
<P><FONT SIZE=2 FACE="Arial">- Based on Python's ElementTree functionality (a minimalist, python specific XML API, see here </FONT><A HREF="http://effbot.org/zone/element-index.htm"><U><FONT COLOR="#0000FF" SIZE=2 FACE="Arial">http://effbot.org/zone/element-index.htm</FONT></U></A><FONT SIZE=2 FACE="Arial"> and here </FONT><A HREF="http://docs.python.org/lib/module-xml.etree.ElementTree.html"><U><FONT COLOR="#0000FF" SIZE=2 FACE="Arial">http://docs.python.org/lib/module-xml.etree.ElementTree.html</FONT></U></A><FONT SIZE=2 FACE="Arial">)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">What I did:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">- Implemented my own parser for ElementTree. The MapFile is parsed into a DOM like structure, and Xpath like expressions can be used to get to nodes, edit their text, etc …</FONT></P>
<P><FONT SIZE=2 FACE="Arial">- Problem: It's new-line sensitive, which it shouldn't be. The benefit is that it is not syntax aware at all, it doesn't care what text content of the map file is, just uses some basic rules around what's expected on a line. (keyword, or keyword + 1-> values).</FONT></P>
<P><FONT SIZE=2 FACE="Arial">The larger, more abstract idea:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">- There have been many debates around XML mapfiles. Implementing something like this for Python, and possibly other languages, could achieve the best of both worlds: Simple, fast, human-friendly MapFiles, that are loadable with XML tools! Most decent XML libraries should allow you to re-implement the parser somewhat, so I have to hope this could be done for Java, Perl, C#, etc … My personal interest is only focused on Python and ElementTree and maybe lxml. Python is good for prototyping anyways :)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Benefits:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Load/Save Mapfile to/from DOM or DOM like structures. Some examples of how this might be useful:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">- Batch processing of mapfiles (find and replace, for example). I've used my implementation to migrate ~500 mapfiles from MapServer 3.6.6 to 4.8 for example. A real life saver.</FONT></P>
<P><FONT SIZE=2 FACE="Arial">- Loading into XML tools/sdks of your liking, to do all those XML things you might like (Editing, validation, transformation, etc …)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">- MapFile editors: GUIs and Web systems for editing of structured documents are nice. wxPython's Tree implementation for example, can be tied to a Document tree. Similar concepts can apply on the web (Think MapStorer)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Multilingual support:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Has this question ever been addressed? One short term interest for us in this is devising a better way to handle multiple languages. Unless there's interest in putting this in the mapserver core, my best idea so far is to use the above mentionned approach to have a master map file, and 2 language specific (en/fr) separate files, and a way to merge and generate 1 mapfile per language.</FONT></P>
<P><FONT SIZE=2 FACE="Arial">The initial challenge:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">The parser shouldn't be line aware. In that case, it means it has to be keyword/token aware, which means that information has to come from somewhere. Right now it's in the lexer definition it looks like. Maybe this could be used by other parser generators, or something like that? TBD …</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Also, defining the exact schema. Once a MapFile has been DOM-ified, should COLOR have a string "255 255 255"? Or elements for "Red", "Green" and "Blue" with separate values. How would this be expressed? An XSD (I think that's what I'd done in my MapFile editor, I'll have to dig it up)? Not at all (Dumb keyword -> value setup)?</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Oh and right, the first thing I'd asked Steve: replace "END" with something matching the opening tag such as "MAPEND" or "CLASSEND" or maybe using Apache conf style "<Map></Map>" (though these would be tokens NOT XML, and the rest of the map file would retain its current syntax).</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Could a language neutral way to define lexical and grammar rules be put in place, that both the C core, and other languages could use?</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Bonus:</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">If I don't work on this before then, I just might get a student in January to work on precisely this!</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Thoughts, comments, constructive criticism?</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Thanks,</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Jean-François Doyon</FONT>
<BR><FONT SIZE=2 FACE="Arial">Data Dissemination Division | Division de la diffusion des données</FONT>
<BR><FONT SIZE=2 FACE="Arial">Data Management and Dissemination Branch | Direction de la gestion et de la diffusion des données</FONT>
<BR><FONT SIZE=2 FACE="Arial">Earth Sciences Sector | Secteur des sciences de la Terre</FONT>
<BR><FONT SIZE=2 FACE="Arial">Natural Resources Canada | Ressources naturelles Canada</FONT>
<BR><FONT SIZE=2 FACE="Arial">Ottawa, Canada K1A 0E9</FONT>
<BR><FONT SIZE=2 FACE="Arial">jdoyon@nrcan-rncan.gc.ca</FONT>
<BR><FONT SIZE=2 FACE="Arial">Telephone | Téléphone 613-992-4902</FONT>
<BR><FONT SIZE=2 FACE="Arial">Facsimile | Télécopieur 613-947-2410</FONT>
<BR><FONT SIZE=2 FACE="Arial">Teletypewriter | Téléimprimeur 613-996-4397</FONT>
<BR><FONT SIZE=2 FACE="Arial">Government of Canada | Gouvernement du Canada</FONT>
</P>
</BODY>
</HTML>