request for comment: adding xslt processing to mapserver
Paul Spencer
pspencer at DMSOLUTIONS.CA
Thu Mar 22 15:50:55 EDT 2007
MapServer devs ...
DMSG has a contract with Environment Canada/Tom Kralidis to implement
some changes to the Sensor Observation Service.
Part of this contract is to write a technical report on the
feasibility of introducing the ability to transform XML coming out of
MapServer's various OGC interfaces (specifically WMS GetFeatureInfo,
WFS GetFeature and SOS GetObservation) using an XSLT directly in
MapServer. I (with help from Assefa) have prepared a draft of this
report structured more-or-less like an RFC.
One of the requirements is to 'gauge the reaction of the TSC to this
proposal', which is the purpose of this email. If the general
feedback from you is positive, I believe Tom will then submit this as
an RFC for approval and, if approved, will seek funding to implement
the changes.
Please give the attached proposal a read over and let me know:
* would you be for or against this capability being introduced into
mapserver?
* if against, would any rewording or additional detail change your mind?
* is there enough technical detail in the proposal?
* is there anything obvious missing (I know I may have missed some
files that would need to be modified)?
* any other comments?
I know this is last minute, but the report is due for mid next week
so getting your feedback ASAP is highly desirable.
Cheers
Paul
+-----------------------------------------------------------------+
|Paul Spencer pspencer at dmsolutions.ca |
+-----------------------------------------------------------------+
|Chief Technology Officer |
|DM Solutions Group Inc http://www.dmsolutions.ca/ |
+-----------------------------------------------------------------+
Mapserver Output Transformation Extension (MOTE)
Motivation
The purpose of MOTE is to allow for the manipulation of XML output
from MapServer through an XSLT transformation. Existing interfaces
that output XML are the WMS GetFeatureInfo (with appropriate mime
type), WFS server and SOS server.
Solutions external to MapServer are possible using other technology.
This can be undesirable for several reasons:
* portability
* performance
* additional points of failure for a service
* additional complexity for IT installation, setup, and maintenance
* potential exposure of the original XML if clients can connect
directly to MapServer
This type of conversion could also be handled in the WxS scripting
interface of mapscript. As with an external solution, this requires
additional complexity, overhead and skill sets beyond the minimum
required to implement this kind of transformation.
Introducing this capability directly in MapServer avoids these problems.
Technical approach
This work will be done by adding the option to include libxslt -
http://xmlsoft.org/XSLT/index.html - in the configure/build stage and
by adding map and/or layer level metadata that specifies an external
XSLT file to use for processing the final output.
The work will require that all existing, related XML output methods
be converted from msIO_printf to libxml2. Note that the SOS Server
already uses libxml2 and would not need to be converted.
In addition to the mandatory changes to functions that output XML
that would need to be transformed, it may be desirable to convert
other functions that output XML to use libxml2 and to share/reuse
common code (such as exception output).
The potential to apply XSLT transformations would be introduced into
WMS GetFeatureInfo (for the appropriate mime types), WFS GetFeature,
and SOS GetObservation service capabilities. In each of these
components, as each layer is processed, the resulting xml node may be
transformed if an XSLT is specified at the layer level. The final
step before outputting the root node as XML would be to apply a map
level transform if an XSLT is specified at the web metadata level.
An XSLT document may be referenced in the file system using either a
relative path (relative to the map file) or an absolute path, or via
a URL.
Error Handling
Suitable error handling will be specified for the following conditions:
* the XSLT document could not be loaded because it does not exist at
the specified location. This should be a non-fatal error condition
that results in a notice or warning being emitted via normal logging
functions
* the XSLT document could not be loaded because it is invalid. This
is caused by a syntax error in the XSLT document that prevents
libxslt from loading the XSLT document. This should be a non-fatal
error condition that results in a notice or warning being emitted via
normal logging functions
* an exception occurred processing the XSLT document. Where
possible, exceptions will be trapped and converted into meaningful
error messages emitted via normal logging channels and processing
should continue with untransformed output.
All error messages shall include as much contextual information and
information from the underlying subsystem as possible to assist in
fixing problems with XSLT transformations.
Additional Libraries
A dependency on libxslt (with a prerequisite to have libxml2) would
be introduced for WMS, WFS and SOS builds.
Files and objects affected
The following files will be affected by this work.
configure/make
The mapserver build system for windows and linux will be modified to
include a dependency on libxslt if appropriately configured.
mapwfs.c
Only msWFSGetFeature would require conversion, although other
functions could be converted for consistency.
mapgml.c
All functions need to be converted. Existing functions that call
msIO_printf would create or add to an xmlNode object and return the
result.
mapwms.c
Only msWMSFeatureInfo would require conversion, although other
functions could be converted for consistency.
mapwcs.c
No functions in mapwcs.c require conversion, but it may be desirable
to convert them for consistency with other OGC services.
Backwards compatibility issues
There are no backwards compatibility issues since adding libxslt is a
build-time configuration option and there are no new or changed
keywords in the mapfile syntax.
Implementation issues
Documentation and examples will be provided as part of the
implementation of this work, in the form of updates to existing How
To documents for WMS, WFS and SOS, and a new How To dealing
specifically with the XSLT component.
There should be mapscript no implications.
There may be build considerations for php_mapscript versions if PHP
is loading libxslt as well.
Test suite
msautotest scripts will be extended to include tests for this
functionality, including normal processing cases (without XSLT) and
each combination of possibilities of XSLT processing including all
known sources of erroneous input.
Examples
Sample XML Output
<?xml version='1.0' encoding="ISO-8859-1" ?>
<owscat:msFeatureCollection
version="1.0.0"
xmlns:owscat="http://www.ec.gc.ca/owscat"
xmlns:gml="http://www.opengis.net/gml"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.ec.gc.ca/owscat http://
devgeo.cciw.ca/cgi-bin/mapserv/owscat?
SERVICE=WFS&VERSION=1.0.0&REQUEST=DescribeFeatureType&TYPENA
ME=service_resources&OUTPUTFORMAT=SFE_XMLSCHEMA">
<gml:boundedBy>
<gml:Envelope srsName="EPSG:4326">
<gml:pos>-180.000000 -90.000000</gml:pos>
<gml:pos>180.000000 90.000000</gml:pos>
</gml:Envelope>
</gml:boundedBy>
<gml:featureMember>
<owscat:service_resources>
<gml:boundedBy>
<gml:Envelope srsName="EPSG:4326">
<gml:pos>-180.000000 -90.000000</gml:pos>
<gml:pos>180.000000 90.000000</gml:pos>
</gml:Envelope>
</gml:boundedBy>
<owscat:msGeometry>
<gml:Polygon srsName="EPSG:4326">
<gml:exterior>
<gml:LinearRing>
<gml:posList srsDimension="2">-180.000000 -90.000000
-180.000000 90.000000 180.000000 90.000000 180.000000 -90.000000
-180.000000 -90.000000 </gml:posList>
</gml:LinearRing>
</gml:exterior>
</gml:Polygon>
</owscat:msGeometry>
<owscat:lang>en-CA</owscat:lang>
<owscat:organization>NASA</owscat:organization>
<owscat:logourl></owscat:logourl>
<owscat:endpoint_getresource>http://aes.gsfc.nasa.gov/cgi-
bin/wms?</owscat:endpoint_getresource>
<owscat:service_type>OGC:WMS</owscat:service_type>
<owscat:service_version>1.1.1</owscat:service_version>
<owscat:resource_id>3572</owscat:resource_id>
<owscat:service_id>111</owscat:service_id>
<owscat:name>2916_21230</owscat:name>
<owscat:title>Complete Earth (2048x1024 Image)</owscat:title>
<owscat:abstract>This image of Earth&#039;s city lights
was created with data from the Defense Meteorological Satellite
Program (DMSP) Operational Linescan System (OLS). Originally designed
to view clouds by moonlight, the OLS is also used to map the
locations of permanent lights on the Earth&#039;s surface.
The brightest areas of the Earth are the most urbanized, but not
necessarily the most populated. (Compare western Europe with China
and India.) Cities tend to grow along coastlines and transportation
networks. Even without the underlying map, the outlines of many
continents would still be visible. The United States interstate
highway system appears as a lattice connecting the brighter dots of
city centers. In Russia, the Trans-Siberian railroad is a thin line
stretching from Moscow through the center of Asia to Vladivostok. The
Nile River, from the Aswan Dam to the Mediterranean Sea, is another
bright thread through an otherwise dark region.
Even more than 100 years after the invention of the electric light,
some regions remain thinly populated and unlit. Antarctica is
entirely dark. The interior jungles of Africa and South America are
mostly dark, but lights are beginning to appear there. Deserts in
Africa, Arabia, Australia, Mongolia, and the United States are poorly
lit as well (except along the coast), along with the boreal forests
of Canada and Russia, and the great mountains of the Himalaya.</
owscat:abstract>
<owscat:keywords>Human geography,Lights,Urbanization,EARTH
SCIENCE:Human Dimensions:Population:Population Distribution</
owscat:keywords>
<owscat:dataurl></owscat:dataurl>
<owscat:metadataurl>http://svs.gsfc.nasa.gov/vis/a000000/
a002900/a002916/a002916.fgdc</owscat:metadataurl>
<owscat:legendurl></owscat:legendurl>
<owscat:scale_min>-99</owscat:scale_min>
<owscat:scale_max>-99</owscat:scale_max>
<owscat:srs>EPSG:4326</owscat:srs>
<owscat:format_list>image/png</owscat:format_list>
<owscat:style_list>opaque</owscat:style_list>
<owscat:time_extent></owscat:time_extent>
<owscat:queryable>0</owscat:queryable>
</owscat:service_resources>
</gml:featureMember>
</owscat:msFeatureCollection>
Sample XLST
<?xml version="1.0" encoding="iso-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/
Transform" xmlns:owscat="http://www.ec.gc.ca/owscat"
xmlns:gml="http://www.opengis.net/gml">
<xsl:output indent="yes" medhod="xml"/>
<xsl:template match="/">
<records>
<xsl:apply-templates/>
</records>
</xsl:template>
<xsl:template match="gml:featureMember/owscat:service_resources">
<record>
<name><xsl:value-of select="owscat:name"/></name>
<title><xsl:value-of select="owscat:title"/></title>
<abstract><xsl:value-of select="owscat:abstract"/></abstract>
</record>
</xsl:template>
</xsl:stylesheet>
Example Result
<?xml version="1.0"?>
<records xmlns:owscat="http://www.ec.gc.ca/owscat" xmlns:gml="http://
www.opengis.net/gml">
<record><name>2916_21230</name><title>Complete Earth
(2048x1024 Image)</title><abstract>This image of Earth&#039;s
city lights was created with data from the Defense Meteorological
Satellite Program (DMSP) Operational Linescan System (OLS).
Originally designed to view clouds by moonlight, the OLS is also used
to map the locations of permanent lights on the Earth&#039;s
surface.
The brightest areas of the Earth are the most urbanized, but not
necessarily the most populated. (Compare western Europe with China
and India.) Cities tend to grow along coastlines and transportation
networks. Even without the underlying map, the outlines of many
continents would still be visible. The United States interstate
highway system appears as a lattice connecting the brighter dots of
city centers. In Russia, the Trans-Siberian railroad is a thin line
stretching from Moscow through the center of Asia to Vladivostok. The
Nile River, from the Aswan Dam to the Mediterranean Sea, is another
bright thread through an otherwise dark region.
Even more than 100 years after the invention of the electric light,
some regions remain thinly populated and unlit. Antarctica is
entirely dark. The interior jungles of Africa and South America are
mostly dark, but lights are beginning to appear there. Deserts in
Africa, Arabia, Australia, Mongolia, and the United States are poorly
lit as well (except along the coast), along with the boreal forests
of Canada and Russia, and the great mountains of the Himalaya.</
abstract></record>
</records>
More information about the mapserver-dev
mailing list