request for comment: adding xslt processing to mapserver
Stephen Woodbridge
woodbri at SWOODBRIDGE.COM
Sat Mar 24 11:14:48 EDT 2007
Paul,
I am somewhat neutral on this proposal, since I don't see a general need
for it. I can see how it might have some benefits for what I think is a
generally small audience.
My basic concerns are:
1) this seems to add significant complexity for the user
- mitigated by optional inclusion
- mitigated partially by the additional documentation
- technically this does not seem overly complex to add
2) limited audience (at least that is my perception)
3) additional drag on the overall development/test/release process
- mitigated partially by the added msautotest
4) Steve L raised some issues around WFS that I would like to see
addressed in the RFC
5) Would you be targeting this for 5.0 release? What if any impact would
you see to the release if this was included?
I don't think I would veto this.
Something that you could add to the RFC that might get me more excited
is something like examples/benefits that are more concrete. Like what
this would give a mapserver user that they don't have today. Adding
embedded XSLT to transform your output is pretty abstract conceptually.
How might this allow a user to do something neat with OpenLayers and WFS
that I can't easily do with the existing code? Or something else along
the lines of showing benefits to the users.
-Steve W
Paul Spencer wrote:
> MapServer devs ...
>
> DMSG has a contract with Environment Canada/Tom Kralidis to implement
> some changes to the Sensor Observation Service.
>
> Part of this contract is to write a technical report on the feasibility
> of introducing the ability to transform XML coming out of MapServer's
> various OGC interfaces (specifically WMS GetFeatureInfo, WFS GetFeature
> and SOS GetObservation) using an XSLT directly in MapServer. I (with
> help from Assefa) have prepared a draft of this report structured
> more-or-less like an RFC.
>
> One of the requirements is to 'gauge the reaction of the TSC to this
> proposal', which is the purpose of this email. If the general feedback
> from you is positive, I believe Tom will then submit this as an RFC for
> approval and, if approved, will seek funding to implement the changes.
>
> Please give the attached proposal a read over and let me know:
>
> * would you be for or against this capability being introduced into
> mapserver?
>
> * if against, would any rewording or additional detail change your mind?
>
> * is there enough technical detail in the proposal?
>
> * is there anything obvious missing (I know I may have missed some files
> that would need to be modified)?
>
> * any other comments?
>
> I know this is last minute, but the report is due for mid next week so
> getting your feedback ASAP is highly desirable.
>
> Cheers
>
> Paul
>
> +-----------------------------------------------------------------+
> |Paul Spencer pspencer at dmsolutions.ca |
> +-----------------------------------------------------------------+
> |Chief Technology Officer |
> |DM Solutions Group Inc http://www.dmsolutions.ca/ |
> +-----------------------------------------------------------------+
>
> Mapserver Output Transformation Extension (MOTE)
>
> Motivation
>
> The purpose of MOTE is to allow for the manipulation of XML output from
> MapServer through an XSLT transformation. Existing interfaces that
> output XML are the WMS GetFeatureInfo (with appropriate mime type), WFS
> server and SOS server.
>
> Solutions external to MapServer are possible using other technology.
> This can be undesirable for several reasons:
>
> * portability
> * performance
> * additional points of failure for a service
> * additional complexity for IT installation, setup, and maintenance
> * potential exposure of the original XML if clients can connect directly
> to MapServer
>
> This type of conversion could also be handled in the WxS scripting
> interface of mapscript. As with an external solution, this requires
> additional complexity, overhead and skill sets beyond the minimum
> required to implement this kind of transformation.
>
> Introducing this capability directly in MapServer avoids these problems.
>
> Technical approach
>
> This work will be done by adding the option to include libxslt -
> http://xmlsoft.org/XSLT/index.html - in the configure/build stage and by
> adding map and/or layer level metadata that specifies an external XSLT
> file to use for processing the final output.
>
> The work will require that all existing, related XML output methods be
> converted from msIO_printf to libxml2. Note that the SOS Server already
> uses libxml2 and would not need to be converted.
>
> In addition to the mandatory changes to functions that output XML that
> would need to be transformed, it may be desirable to convert other
> functions that output XML to use libxml2 and to share/reuse common code
> (such as exception output).
>
> The potential to apply XSLT transformations would be introduced into WMS
> GetFeatureInfo (for the appropriate mime types), WFS GetFeature, and SOS
> GetObservation service capabilities. In each of these components, as
> each layer is processed, the resulting xml node may be transformed if an
> XSLT is specified at the layer level. The final step before outputting
> the root node as XML would be to apply a map level transform if an XSLT
> is specified at the web metadata level.
>
> An XSLT document may be referenced in the file system using either a
> relative path (relative to the map file) or an absolute path, or via a URL.
>
> Error Handling
>
> Suitable error handling will be specified for the following conditions:
>
> * the XSLT document could not be loaded because it does not exist at the
> specified location. This should be a non-fatal error condition that
> results in a notice or warning being emitted via normal logging functions
> * the XSLT document could not be loaded because it is invalid. This is
> caused by a syntax error in the XSLT document that prevents libxslt from
> loading the XSLT document. This should be a non-fatal error condition
> that results in a notice or warning being emitted via normal logging
> functions
> * an exception occurred processing the XSLT document. Where possible,
> exceptions will be trapped and converted into meaningful error messages
> emitted via normal logging channels and processing should continue with
> untransformed output.
>
> All error messages shall include as much contextual information and
> information from the underlying subsystem as possible to assist in
> fixing problems with XSLT transformations.
>
> Additional Libraries
>
> A dependency on libxslt (with a prerequisite to have libxml2) would be
> introduced for WMS, WFS and SOS builds.
>
> Files and objects affected
>
> The following files will be affected by this work.
>
> configure/make
>
> The mapserver build system for windows and linux will be modified to
> include a dependency on libxslt if appropriately configured.
>
> mapwfs.c
>
> Only msWFSGetFeature would require conversion, although other functions
> could be converted for consistency.
>
> mapgml.c
>
> All functions need to be converted. Existing functions that call
> msIO_printf would create or add to an xmlNode object and return the result.
>
> mapwms.c
>
> Only msWMSFeatureInfo would require conversion, although other functions
> could be converted for consistency.
>
> mapwcs.c
>
> No functions in mapwcs.c require conversion, but it may be desirable to
> convert them for consistency with other OGC services.
>
> Backwards compatibility issues
>
> There are no backwards compatibility issues since adding libxslt is a
> build-time configuration option and there are no new or changed keywords
> in the mapfile syntax.
>
> Implementation issues
>
> Documentation and examples will be provided as part of the
> implementation of this work, in the form of updates to existing How To
> documents for WMS, WFS and SOS, and a new How To dealing specifically
> with the XSLT component.
>
> There should be mapscript no implications.
>
> There may be build considerations for php_mapscript versions if PHP is
> loading libxslt as well.
>
> Test suite
>
> msautotest scripts will be extended to include tests for this
> functionality, including normal processing cases (without XSLT) and each
> combination of possibilities of XSLT processing including all known
> sources of erroneous input.
>
> Examples
>
> Sample XML Output
>
> <?xml version='1.0' encoding="ISO-8859-1" ?>
> <owscat:msFeatureCollection
> version="1.0.0"
> xmlns:owscat="http://www.ec.gc.ca/owscat"
> xmlns:gml="http://www.opengis.net/gml"
> xmlns:ogc="http://www.opengis.net/ogc"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.ec.gc.ca/owscat
> http://devgeo.cciw.ca/cgi-bin/mapserv/owscat?SERVICE=WFS&VERSION=1.0.0&REQUEST=DescribeFeatureType&TYPENAME=service_resources&OUTPUTFORMAT=SFE_XMLSCHEMA">
>
> <gml:boundedBy>
> <gml:Envelope srsName="EPSG:4326">
> <gml:pos>-180.000000 -90.000000</gml:pos>
> <gml:pos>180.000000 90.000000</gml:pos>
> </gml:Envelope>
> </gml:boundedBy>
> <gml:featureMember>
>
> <owscat:service_resources>
> <gml:boundedBy>
> <gml:Envelope srsName="EPSG:4326">
> <gml:pos>-180.000000 -90.000000</gml:pos>
> <gml:pos>180.000000 90.000000</gml:pos>
> </gml:Envelope>
> </gml:boundedBy>
> <owscat:msGeometry>
>
> <gml:Polygon srsName="EPSG:4326">
> <gml:exterior>
> <gml:LinearRing>
> <gml:posList srsDimension="2">-180.000000 -90.000000
> -180.000000 90.000000 180.000000 90.000000 180.000000 -90.000000
> -180.000000 -90.000000 </gml:posList>
> </gml:LinearRing>
> </gml:exterior>
> </gml:Polygon>
> </owscat:msGeometry>
>
> <owscat:lang>en-CA</owscat:lang>
> <owscat:organization>NASA</owscat:organization>
> <owscat:logourl></owscat:logourl>
>
> <owscat:endpoint_getresource>http://aes.gsfc.nasa.gov/cgi-bin/wms?</owscat:endpoint_getresource>
>
> <owscat:service_type>OGC:WMS</owscat:service_type>
> <owscat:service_version>1.1.1</owscat:service_version>
>
> <owscat:resource_id>3572</owscat:resource_id>
> <owscat:service_id>111</owscat:service_id>
> <owscat:name>2916_21230</owscat:name>
> <owscat:title>Complete Earth (2048x1024 Image)</owscat:title>
> <owscat:abstract>This image of Earth&#039;s city lights was
> created with data from the Defense Meteorological Satellite Program
> (DMSP) Operational Linescan System (OLS). Originally designed to view
> clouds by moonlight, the OLS is also used to map the locations of
> permanent lights on the Earth&#039;s surface.
> The brightest areas of the Earth are the most urbanized, but not
> necessarily the most populated. (Compare western Europe with China and
> India.) Cities tend to grow along coastlines and transportation
> networks. Even without the underlying map, the outlines of many
> continents would still be visible. The United States interstate highway
> system appears as a lattice connecting the brighter dots of city
> centers. In Russia, the Trans-Siberian railroad is a thin line
> stretching from Moscow through the center of Asia to Vladivostok. The
> Nile River, from the Aswan Dam to the Mediterranean Sea, is another
> bright thread through an otherwise dark region.
> Even more than 100 years after the invention of the electric light, some
> regions remain thinly populated and unlit. Antarctica is entirely dark.
> The interior jungles of Africa and South America are mostly dark, but
> lights are beginning to appear there. Deserts in Africa, Arabia,
> Australia, Mongolia, and the United States are poorly lit as well
> (except along the coast), along with the boreal forests of Canada and
> Russia, and the great mountains of the Himalaya.</owscat:abstract>
>
> <owscat:keywords>Human geography,Lights,Urbanization,EARTH
> SCIENCE:Human Dimensions:Population:Population
> Distribution</owscat:keywords>
> <owscat:dataurl></owscat:dataurl>
>
> <owscat:metadataurl>http://svs.gsfc.nasa.gov/vis/a000000/a002900/a002916/a002916.fgdc</owscat:metadataurl>
>
> <owscat:legendurl></owscat:legendurl>
> <owscat:scale_min>-99</owscat:scale_min>
> <owscat:scale_max>-99</owscat:scale_max>
> <owscat:srs>EPSG:4326</owscat:srs>
>
> <owscat:format_list>image/png</owscat:format_list>
> <owscat:style_list>opaque</owscat:style_list>
> <owscat:time_extent></owscat:time_extent>
> <owscat:queryable>0</owscat:queryable>
> </owscat:service_resources>
> </gml:featureMember>
> </owscat:msFeatureCollection>
>
> Sample XLST
>
> <?xml version="1.0" encoding="iso-8859-1"?>
> <xsl:stylesheet version="1.0"
> xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
> xmlns:owscat="http://www.ec.gc.ca/owscat"
> xmlns:gml="http://www.opengis.net/gml">
>
> <xsl:output indent="yes" medhod="xml"/>
>
> <xsl:template match="/">
> <records>
> <xsl:apply-templates/>
> </records>
> </xsl:template>
>
> <xsl:template match="gml:featureMember/owscat:service_resources">
> <record>
> <name><xsl:value-of select="owscat:name"/></name>
> <title><xsl:value-of select="owscat:title"/></title>
> <abstract><xsl:value-of select="owscat:abstract"/></abstract>
> </record>
> </xsl:template>
>
> </xsl:stylesheet>
>
> Example Result
>
> <?xml version="1.0"?>
> <records xmlns:owscat="http://www.ec.gc.ca/owscat"
> xmlns:gml="http://www.opengis.net/gml">
> <record><name>2916_21230</name><title>Complete Earth (2048x1024
> Image)</title><abstract>This image of Earth&#039;s city lights was
> created with data from the Defense Meteorological Satellite Program
> (DMSP) Operational Linescan System (OLS). Originally designed to view
> clouds by moonlight, the OLS is also used to map the locations of
> permanent lights on the Earth&#039;s surface.
> The brightest areas of the Earth are the most urbanized, but not
> necessarily the most populated. (Compare western Europe with China and
> India.) Cities tend to grow along coastlines and transportation
> networks. Even without the underlying map, the outlines of many
> continents would still be visible. The United States interstate highway
> system appears as a lattice connecting the brighter dots of city
> centers. In Russia, the Trans-Siberian railroad is a thin line
> stretching from Moscow through the center of Asia to Vladivostok. The
> Nile River, from the Aswan Dam to the Mediterranean Sea, is another
> bright thread through an otherwise dark region.
> Even more than 100 years after the invention of the electric light, some
> regions remain thinly populated and unlit. Antarctica is entirely dark.
> The interior jungles of Africa and South America are mostly dark, but
> lights are beginning to appear there. Deserts in Africa, Arabia,
> Australia, Mongolia, and the United States are poorly lit as well
> (except along the coast), along with the boreal forests of Canada and
> Russia, and the great mountains of the Himalaya.</abstract></record>
> </records>
More information about the mapserver-dev
mailing list