request for comment: adding xslt processing to mapserver
Kralidis,Tom [Burlington]
Tom.Kralidis at EC.GC.CA
Mon Mar 26 11:25:34 EDT 2007
Comments inline:
> -----Original Message-----
> From: UMN MapServer Developers List
> [mailto:MAPSERVER-DEV at LISTS.UMN.EDU] On Behalf Of Paul Spencer
> Sent: 24 March, 2007 5:24 PM
> To: MAPSERVER-DEV at LISTS.UMN.EDU
> Subject: *****SPAM***** Re: [UMN_MAPSERVER-DEV] request for
> comment: adding xslt processing to mapserver
>
> Thanks for your feedback, Steve. A couple of comments inline.
>
> On 24-Mar-07, at 11:14 AM, Stephen Woodbridge wrote:
>
> > Paul,
> >
> > I am somewhat neutral on this proposal, since I don't see a general
> > need for it. I can see how it might have some benefits for what I
> > think is a generally small audience.
> >
> > My basic concerns are:
> >
> > 1) this seems to add significant complexity for the user
> > - mitigated by optional inclusion
> > - mitigated partially by the additional documentation
> > - technically this does not seem overly complex to add
>
> only for those that want to use it. I would consider this a
> feature that is not for the faint of heart!
>
> > 2) limited audience (at least that is my perception)
>
> Anyone using MapServer for OGC services could be potentially
> benefited. I'm not sure how many people that is, but its a
> non- trivial number :)
>
> > 3) additional drag on the overall development/test/release process
> > - mitigated partially by the added msautotest
>
> Not sure I understand your concern on this? Do you think it
> would be more difficult to build code using libxml2 rather
> than using msIO_printf? I believe that it will result in
> less erroneous output since the structure and formatting of
> the output will be handled in a consistent and reliable
> manner. The current approach is a ripe breeding ground for
> all kinds of problems. Once there is a sufficient base of
> code built on libxml2, I think it will be quite easy to work
> with for most of the core developers.
>
msIO_printf vs. libxml2: I think this is an underlying issue of
importance. We tried libxml2 when we implemented OGC:SOS, and found the
library was easy to use (even I understand it), and takes a lot of the
heartache out of writing XML (well-formedness). Check out mapogcsos.c
and mapowscommon.c for libxml2 usage.
> > 4) Steve L raised some issues around WFS that I would like to see
> > addressed in the RFC
>
> I am adding notes about:
>
> * limitations of using XSLT (entire doc in memory)
> * requiring incoming transformations for WFS GetFeature
> * that schemas for transformed WFS GetFeature results would
> be handled by the implementor (i.e. they provide the xslt or
> external reference to the schema)
>
> > 5) Would you be targeting this for 5.0 release? What if any impact
> > would you see to the release if this was included?
>
> Given that MapServer 5.0 is currently looking at a May
> timeframe, I doubt this would be targetted for 5.0. This
> effort is to build a
> reasonable case that can be used to obtain funding post March 31st.
> Given the way funding and budgets go, I would suspect it would be
> 6-12 months before this would be brought to implementation
> assuming it is technically approved.
>
> >
> > I don't think I would veto this.
> >
> > Something that you could add to the RFC that might get me
> more excited
> > is something like examples/benefits that are more concrete.
> > Like what this would give a mapserver user that they don't
> have today.
> > Adding embedded XSLT to transform your output is pretty abstract
> > conceptually.
> >
> > How might this allow a user to do something neat with
> OpenLayers and
> > WFS that I can't easily do with the existing code? Or
> something else
> > along the lines of showing benefits to the users.
>
> Good points. I won't have anything like this before I hand
> the results off, but I will include a recommendation that
> concrete examples be provided before submitting the RFC.
>
> >
> > -Steve W
> >
> > Paul Spencer wrote:
> >> MapServer devs ...
> >> DMSG has a contract with Environment Canada/Tom Kralidis
> to implement
> >> some changes to the Sensor Observation Service.
> >> Part of this contract is to write a technical report on the
> >> feasibility of introducing the ability to transform XML
> coming out of
> >> MapServer's various OGC interfaces (specifically WMS
> GetFeatureInfo,
> >> WFS GetFeature and SOS GetObservation) using an XSLT directly in
> >> MapServer. I (with help from Assefa) have prepared a
> draft of this
> >> report structured more-or-less like an RFC.
> >> One of the requirements is to 'gauge the reaction of the
> TSC to this
> >> proposal', which is the purpose of this email. If the general
> >> feedback from you is positive, I believe Tom will then
> submit this as
> >> an RFC for approval and, if approved, will seek funding to
> implement
> >> the changes.
> >> Please give the attached proposal a read over and let me know:
> >> * would you be for or against this capability being
> introduced into
> >> mapserver?
> >> * if against, would any rewording or additional detail change your
> >> mind?
> >> * is there enough technical detail in the proposal?
> >> * is there anything obvious missing (I know I may have missed some
> >> files that would need to be modified)?
> >> * any other comments?
> >> I know this is last minute, but the report is due for mid
> next week
> >> so getting your feedback ASAP is highly desirable.
> >> Cheers
> >> Paul
> >> +-----------------------------------------------------------------+
> >> |Paul Spencer pspencer at dmsolutions.ca |
> >> +-----------------------------------------------------------------+
> >> |Chief Technology Officer |
> >> |DM Solutions Group Inc http://www.dmsolutions.ca/ |
> >> +-----------------------------------------------------------------+
> >> Mapserver Output Transformation Extension (MOTE) Motivation The
> >> purpose of MOTE is to allow for the manipulation of XML
> output from
> >> MapServer through an XSLT transformation. Existing
> interfaces that
> >> output XML are the WMS GetFeatureInfo (with appropriate
> mime type),
> >> WFS server and SOS server.
> >> Solutions external to MapServer are possible using other
> technology.
> >> This can be undesirable for several reasons:
> >> * portability
> >> * performance
> >> * additional points of failure for a service
> >> * additional complexity for IT installation, setup, and maintenance
> >> * potential exposure of the original XML if clients can connect
> >> directly to MapServer This type of conversion could also
> be handled
> >> in the WxS scripting
> >> interface of mapscript. As with an external solution, this
> >> requires additional complexity, overhead and skill sets beyond the
> >> minimum required to implement this kind of transformation.
> >> Introducing this capability directly in MapServer avoids these
> >> problems.
> >> Technical approach
> >> This work will be done by adding the option to include libxslt -
> >> http://xmlsoft.org/XSLT/index.html - in the
> configure/build stage and
> >> by adding map and/or layer level metadata that specifies
> an external
> >> XSLT file to use for processing the final output.
> >> The work will require that all existing, related XML
> output methods
> >> be converted from msIO_printf to libxml2. Note that the
> SOS Server
> >> already uses libxml2 and would not need to be converted.
> >> In addition to the mandatory changes to functions that output XML
> >> that would need to be transformed, it may be desirable to convert
> >> other functions that output XML to use libxml2 and to share/reuse
> >> common code (such as exception output).
> >> The potential to apply XSLT transformations would be
> introduced into
> >> WMS GetFeatureInfo (for the appropriate mime types), WFS
> GetFeature,
> >> and SOS GetObservation service capabilities. In each of these
> >> components, as each layer is processed, the resulting xml
> node may be
> >> transformed if an XSLT is specified at the layer level. The final
> >> step before outputting the root node as XML would be to
> apply a map
> >> level transform if an XSLT is specified at the web metadata level.
> >> An XSLT document may be referenced in the file system
> using either a
> >> relative path (relative to the map file) or an absolute
> path, or via
> >> a URL.
> >> Error Handling
> >> Suitable error handling will be specified for the following
> >> conditions:
> >> * the XSLT document could not be loaded because it does
> not exist at
> >> the specified location. This should be a non-fatal error
> condition
> >> that results in a notice or warning being emitted via
> normal logging
> >> functions
> >> * the XSLT document could not be loaded because it is invalid.
> >> This is caused by a syntax error in the XSLT document that
> prevents
> >> libxslt from loading the XSLT document. This should be a
> non-fatal
> >> error condition that results in a notice or warning being
> emitted via
> >> normal logging functions
> >> * an exception occurred processing the XSLT document. Where
> >> possible, exceptions will be trapped and converted into meaningful
> >> error messages emitted via normal logging channels and processing
> >> should continue with untransformed output.
> >> All error messages shall include as much contextual
> information and
> >> information from the underlying subsystem as possible to assist in
> >> fixing problems with XSLT transformations.
> >> Additional Libraries
> >> A dependency on libxslt (with a prerequisite to have
> libxml2) would
> >> be introduced for WMS, WFS and SOS builds.
> >> Files and objects affected
> >> The following files will be affected by this work.
> >> configure/make
> >> The mapserver build system for windows and linux will be
> modified to
> >> include a dependency on libxslt if appropriately configured.
> >> mapwfs.c
> >> Only msWFSGetFeature would require conversion, although other
> >> functions could be converted for consistency.
> >> mapgml.c
> >> All functions need to be converted. Existing functions that call
> >> msIO_printf would create or add to an xmlNode object and
> return the
> >> result.
> >> mapwms.c
> >> Only msWMSFeatureInfo would require conversion, although other
> >> functions could be converted for consistency.
> >> mapwcs.c
> >> No functions in mapwcs.c require conversion, but it may be
> desirable
> >> to convert them for consistency with other OGC services.
> >> Backwards compatibility issues
> >> There are no backwards compatibility issues since adding
> libxslt is a
> >> build-time configuration option and there are no new or changed
> >> keywords in the mapfile syntax.
> >> Implementation issues
> >> Documentation and examples will be provided as part of the
> >> implementation of this work, in the form of updates to
> existing How
> >> To documents for WMS, WFS and SOS, and a new How To dealing
> >> specifically with the XSLT component.
> >> There should be mapscript no implications.
> >> There may be build considerations for php_mapscript
> versions if PHP
> >> is loading libxslt as well.
> >> Test suite
> >> msautotest scripts will be extended to include tests for this
> >> functionality, including normal processing cases (without
> XSLT) and
> >> each combination of possibilities of XSLT processing including all
> >> known sources of erroneous input.
> >> Examples
> >> Sample XML Output
> >> <?xml version='1.0' encoding="ISO-8859-1" ?>
> >> <owscat:msFeatureCollection
> >> version="1.0.0"
> >> xmlns:owscat="http://www.ec.gc.ca/owscat"
> >> xmlns:gml="http://www.opengis.net/gml"
> >> xmlns:ogc="http://www.opengis.net/ogc"
> >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >> xsi:schemaLocation="http://www.ec.gc.ca/owscat http://
> >> devgeo.cciw.ca/cgi-bin/mapserv/owscat?
> >>
> SERVICE=WFS&VERSION=1.0.0&REQUEST=DescribeFeatureType&TYP
> >> ENAME=service_resources&OUTPUTFORMAT=SFE_XMLSCHEMA">
> >> <gml:boundedBy>
> >> <gml:Envelope srsName="EPSG:4326">
> >> <gml:pos>-180.000000 -90.000000</gml:pos>
> >> <gml:pos>180.000000 90.000000</gml:pos>
> >> </gml:Envelope>
> >> </gml:boundedBy>
> >> <gml:featureMember>
> >> <owscat:service_resources>
> >> <gml:boundedBy>
> >> <gml:Envelope srsName="EPSG:4326">
> >> <gml:pos>-180.000000 -90.000000</gml:pos>
> >> <gml:pos>180.000000 90.000000</gml:pos>
> >> </gml:Envelope>
> >> </gml:boundedBy>
> >> <owscat:msGeometry>
> >> <gml:Polygon srsName="EPSG:4326">
> >> <gml:exterior>
> >> <gml:LinearRing>
> >> <gml:posList srsDimension="2">-180.000000
> -90.000000
> >> -180.000000 90.000000 180.000000 90.000000 180.000000 -90.000000
> >> -180.000000 -90.000000 </gml:posList>
> >> </gml:LinearRing>
> >> </gml:exterior>
> >> </gml:Polygon>
> >> </owscat:msGeometry>
> >> <owscat:lang>en-CA</owscat:lang>
> >> <owscat:organization>NASA</owscat:organization>
> >> <owscat:logourl></owscat:logourl>
> >> <owscat:endpoint_getresource>http://aes.gsfc.nasa.gov/cgi-
> >> bin/wms?</owscat:endpoint_getresource>
> >> <owscat:service_type>OGC:WMS</owscat:service_type>
> >> <owscat:service_version>1.1.1</owscat:service_version>
> >> <owscat:resource_id>3572</owscat:resource_id>
> >> <owscat:service_id>111</owscat:service_id>
> >> <owscat:name>2916_21230</owscat:name>
> >> <owscat:title>Complete Earth (2048x1024
> Image)</owscat:title>
> >> <owscat:abstract>This image of Earth&#039;s
> city lights
> >> was created with data from the Defense Meteorological Satellite
> >> Program (DMSP) Operational Linescan System (OLS).
> >> Originally designed to view clouds by moonlight, the OLS
> is also used
> >> to map the locations of permanent lights on the Earth&#039;s
> >> surface.
> >> The brightest areas of the Earth are the most urbanized, but not
> >> necessarily the most populated. (Compare western Europe with China
> >> and India.) Cities tend to grow along coastlines and
> transportation
> >> networks. Even without the underlying map, the outlines of many
> >> continents would still be visible. The United States interstate
> >> highway system appears as a lattice connecting the
> brighter dots of
> >> city centers. In Russia, the Trans-Siberian railroad is a
> thin line
> >> stretching from Moscow through the center of Asia to
> Vladivostok. The
> >> Nile River, from the Aswan Dam to the Mediterranean Sea,
> is another
> >> bright thread through an otherwise dark region.
> >> Even more than 100 years after the invention of the
> electric light,
> >> some regions remain thinly populated and unlit. Antarctica is
> >> entirely dark. The interior jungles of Africa and South
> America are
> >> mostly dark, but lights are beginning to appear there. Deserts in
> >> Africa, Arabia, Australia, Mongolia, and the United States
> are poorly
> >> lit as well (except along the coast), along with the
> boreal forests
> >> of Canada and Russia, and the great mountains of the
> >> Himalaya.</owscat:abstract>
> >> <owscat:keywords>Human geography,Lights,Urbanization,EARTH
> >> SCIENCE:Human Dimensions:Population:Population Distribution</
> >> owscat:keywords>
> >> <owscat:dataurl></owscat:dataurl>
> >> <owscat:metadataurl>http://svs.gsfc.nasa.gov/vis/a000000/
> >> a002900/a002916/a002916.fgdc</owscat:metadataurl>
> >> <owscat:legendurl></owscat:legendurl>
> >> <owscat:scale_min>-99</owscat:scale_min>
> >> <owscat:scale_max>-99</owscat:scale_max>
> >> <owscat:srs>EPSG:4326</owscat:srs>
> >> <owscat:format_list>image/png</owscat:format_list>
> >> <owscat:style_list>opaque</owscat:style_list>
> >> <owscat:time_extent></owscat:time_extent>
> >> <owscat:queryable>0</owscat:queryable>
> >> </owscat:service_resources>
> >> </gml:featureMember>
> >> </owscat:msFeatureCollection>
> >> Sample XLST
> >> <?xml version="1.0" encoding="iso-8859-1"?> <xsl:stylesheet
> >> version="1.0" xmlns:xsl="http://www.w3.org/1999/
> >> XSL/Transform" xmlns:owscat="http://www.ec.gc.ca/owscat"
> >> xmlns:gml="http://www.opengis.net/gml">
> >> <xsl:output indent="yes" medhod="xml"/> <xsl:template match="/">
> >> <records>
> >> <xsl:apply-templates/>
> >> </records>
> >> </xsl:template>
> >> <xsl:template match="gml:featureMember/owscat:service_resources">
> >> <record>
> >> <name><xsl:value-of select="owscat:name"/></name>
> >> <title><xsl:value-of select="owscat:title"/></title>
> >> <abstract><xsl:value-of select="owscat:abstract"/></abstract>
> >> </record>
> >> </xsl:template>
> >> </xsl:stylesheet>
> >> Example Result
> >> <?xml version="1.0"?>
> >> <records xmlns:owscat="http://www.ec.gc.ca/owscat"
> >> xmlns:gml="http://www.opengis.net/gml">
> >> <record><name>2916_21230</name><title>Complete Earth
> >> (2048x1024 Image)</title><abstract>This image of Earth&#039;s
> >> city lights was created with data from the Defense Meteorological
> >> Satellite Program (DMSP) Operational Linescan System (OLS).
> >> Originally designed to view clouds by moonlight, the OLS
> is also used
> >> to map the locations of permanent lights on the Earth&#039;s
> >> surface.
> >> The brightest areas of the Earth are the most urbanized, but not
> >> necessarily the most populated. (Compare western Europe with China
> >> and India.) Cities tend to grow along coastlines and
> transportation
> >> networks. Even without the underlying map, the outlines of many
> >> continents would still be visible. The United States interstate
> >> highway system appears as a lattice connecting the
> brighter dots of
> >> city centers. In Russia, the Trans-Siberian railroad is a
> thin line
> >> stretching from Moscow through the center of Asia to
> Vladivostok. The
> >> Nile River, from the Aswan Dam to the Mediterranean Sea,
> is another
> >> bright thread through an otherwise dark region.
> >> Even more than 100 years after the invention of the
> electric light,
> >> some regions remain thinly populated and unlit. Antarctica is
> >> entirely dark. The interior jungles of Africa and South
> America are
> >> mostly dark, but lights are beginning to appear there. Deserts in
> >> Africa, Arabia, Australia, Mongolia, and the United States
> are poorly
> >> lit as well (except along the coast), along with the
> boreal forests
> >> of Canada and Russia, and the great mountains of the
> >> Himalaya.</abstract></record> </records>
> >
>
> +-----------------------------------------------------------------+
> |Paul Spencer pspencer at dmsolutions.ca |
> +-----------------------------------------------------------------+
> |Chief Technology Officer |
> |DM Solutions Group Inc http://www.dmsolutions.ca/ |
> +-----------------------------------------------------------------+
>
More information about the mapserver-dev
mailing list