request for comment: adding xslt processing to mapserver

[Paul Spencer]

Thanks Daniel, I'll incorporate your comments into my report.

I now think this feature is actually two features, using libxml2 and  
then introducing libxslt.  If mapserver already used libxml2 or  
equivalent for handing xml, then this would be a relatively minor  
addition.

One thing that has occurred to me that might provide an alternate way  
of approaching this.

Would it be possible to optionally accumulate the output from  
msIO_printf into an in-memory buffer and then use libxml2 to parse it  
and libxslt to transform it?  I see that msIO_printf gets stdout via  
msIO_getHandler which I would assume would allow directing stdout to  
a memory buffer of some kind?

If this could be done, then we could avoid using libxml2 by default  
and thus not affect, in any way, the normal output from WMS, WFS  
etc.  If an XSLT was requested in this particular case, then the  
calls to msIO_printf would write to a buffer rather than stdout and  
the buffer could be processed through libxml2 and libxslt.

Is this feasible?

Paul

On 28-Mar-07, at 11:06 AM, Daniel Morissette wrote:

> Sorry for the late reply, I know you already added notes about this  
> in your document but I just wanted to add that I think Steve has  
> some very good points here and I am also worried about the  
> performance impact of a complete switch to libxml2, especially on  
> large XML responses (e.g. large GML responses, etc.).
>
> Another point that has not been mentioned yet is that converting  
> all OGC-related code from msIO_printf to libxml2 comes with a high  
> risk of introducing new bugs in something that's currently very  
> stable (I'm mostly thinking about the WMS support). Yes I agree  
> that using printf to generate XML is clunky, but this proposal will  
> take thousands of lines of stable and tested code and rewrite them  
> to use libxml2. This is likely to introduce new bugs in the OGC  
> specs support and this should be taken into account as you plan  
> this proposal.
>
> Daniel
>
>
> Steve Lime wrote:
>> Paul: I did look at libxml2/libxslt and it's XSLT support before  
>> doing any of the work on GML transformation last summer. At that  
>> time I decided against it because:
>>  - performance, libxslt required an entire document be memory  
>> resident
>> before transforming. You had to write vanilla GML to memory first  
>> (as DOM
>> object) and for large datasets that wouldn't be straightforward).  
>> Streaming transformations were  needed and, at the time, only  
>> available via java (and experimental at that), I don't know if  
>> that has changed or not.
>>  - with WFS, what you produce you need to document. So for the  
>> transformed
>> output you'd need to produce a corresponding schema, yet another  
>> XSLT file.
>> Referencing an external schema (e.g. some standard application  
>> schema) is
>> one workaround but I don't know that it is a perfect solution.
>>  - with WFS, what you produce you also need to support queries  
>> against, so
>> now you need to transform an incoming filter into something that  
>> that can be
>> applied against the native data store, basically inverting the  
>> transformation applied in the first place. Then you also have to  
>> deal with non-xml request encoding.
>> In the end it was all too much to swallow, not to mention that  
>> XSLT isn't exactly
>> trivial for users. Not that configuring GML transformations  
>> through metadata is
>> easy- it ain't, it sucks and overly complicates a mapfile. Still  
>> there's an elegance to only having to write construct (and  
>> support!) vanilla GML and then using external transformations. If  
>> the issues above could be addressed
>> I would support the idea and might go one step further and suggest  
>> yanking out
>> the metadata configured tranformation.
>> Steve
>>>>> Paul Spencer <pspencer at DMSOLUTIONS.CA> 03/22/07 2:50 PM >>>
>> MapServer devs ...
>> DMSG has a contract with Environment Canada/Tom Kralidis to  
>> implement  some changes to the Sensor Observation Service.
>> Part of this contract is to write a technical report on the   
>> feasibility of introducing the ability to transform XML coming out  
>> of  MapServer's various OGC interfaces (specifically WMS  
>> GetFeatureInfo,  WFS GetFeature and SOS GetObservation) using an  
>> XSLT directly in  MapServer.  I (with help from Assefa) have  
>> prepared a draft of this  report structured more-or-less like an RFC.
>> One of the requirements is to 'gauge the reaction of the TSC to  
>> this  proposal', which is the purpose of this email. If the  
>> general  feedback from you is positive, I believe Tom will then  
>> submit this as  an RFC for approval and, if approved, will seek  
>> funding to implement  the changes.
>> Please give the attached proposal a read over and let me know:
>> * would you be for or against this capability being introduced  
>> into  mapserver?
>> * if against, would any rewording or additional detail change your  
>> mind?
>> * is there enough technical detail in the proposal?
>> * is there anything obvious missing (I know I may have missed  
>> some  files that would need to be modified)?
>> * any other comments?
>> I know this is last minute, but the report is due for mid next  
>> week  so getting your feedback ASAP is highly desirable.
>> Cheers
>> Paul
>> +-----------------------------------------------------------------+
>> |Paul Spencer                          pspencer at dmsolutions.ca    |
>> +-----------------------------------------------------------------+
>> |Chief Technology Officer                                         |
>> |DM Solutions Group Inc                http://www.dmsolutions.ca/ |
>> +-----------------------------------------------------------------+
>
> -- 
> Daniel Morissette
> http://www.mapgears.com/

+-----------------------------------------------------------------+
|Paul Spencer                          pspencer at dmsolutions.ca    |
+-----------------------------------------------------------------+
|Chief Technology Officer                                         |
|DM Solutions Group Inc                http://www.dmsolutions.ca/ |
+-----------------------------------------------------------------+