RFC 16: MapScript WxS Services

[Frank Warmerdam]

Stephen Woodbridge wrote:
> Frank,
> 
> On exposing the gdbuffer to mapscript via the msIO functions, what
> exactly is getting exposed? Looks like just a pointer to the gd library
> image buffer in memory.

Steve,

The gdBuffer structure currently is just a pointer, and a buffer size
value.  It is used as a container to refer to a sized chunk of binary
memory when returning it to mapscript.

In the previous use of the gdBuffer it was used to provide access
the the gdImage's image buffer, and so it was named gdBuffer.  But
it is really just a general container for a block of raw bytes.

> How can it be manipulated from mapscript?
> I see you example for python. Are planing on adding anything similar for
> the other bindings? Have you requested the other binding owners to add
> support if you do not plan to?

It appears that C#, Java and Python already have SWIG typemaps for
gdBuffer translating it as a binary buffer as appropriate in the
different languages.  I noticed that Perl does not.  I looked into adding
this briefly but my understanding of the perl C API is zero, so I gave
up quickly.  Basically, the gdBuffer stuff already existed as a way of
returning binary data, and I'm piggy backing on it.  It's up to individual
mapscript language owners to support it if they wish, which several already
do.

> For example in Perl there is a GD.pm module (see:
> http://stein.cshl.org/WWW/software/GD ) for more info, would/will there
> be a way to "link up" the buffer with that for additional processing. I
> guess my concern is that getting access to the buffer is not real useful
> unless there are appropriate tools to do something to the buffer.

Agreed, and that really depends on swig typemaps turning the gdBuffer
into something usable as a binary buffer in the host language.

> I see this raised in the Implementation Issues sections but I requesting
> help from the other maintainers would be appropriate if you are unable
> to do it yourself.

Agreed.  Help is welcome.

Note that many of the useful cases for WxS services relate to manipulating
XML and GML responses and don't require binary buffers.  The binary buffers
are primarily helpful for the results of a GetMap call for instance.

>> Questions:
>>
>> 1) Should we be "pushing" handlers instead of installing them and
>> losing track of the previous handler? Then we could just pop them
>> off.
> 
> It seems like pushing, or chaining if you want to have multiple handlers 
> editing an io stream, would be appropriate. Or you could have the method 
> to install handler and return the previous one and then force the 
> programmer to restore it if required, like:
> 
>   oldHandler = mapscript.msIO_installStdoutToBuffer()
>     ...
>   mapscript.msIO_installHandler(oldHandler);
> 
> but this is not as nice as pushing and popping handlers or filters. An 
> option could be to install filters on the various streams that could 
> manipulate the data and even redirect it if required. You are already 
> doing that but only for a single handler. Something like:

I hadn't really conceived of this as filters, though I suppose this
is possible.

Note, I've deliberately "dumbed down" the lower level msIO interface
for installing handlers for use from MapScript.  The low level API
provides access to handler function pointers, "void *" callback
data and so forth.  But I tried to prepare a simplified interface
for mapscript so that we won't be stuck trying to cope with handling
function pointers and the such in each of the mapscript languages.

>   sub myfilter {
>     ...
>   }
> 
>   mapscript.msIO_AddFilter(MS_STDOUT, \&my_filter)
>     ...
>   mapscript.msIO_RemoveFilter(MS_STDOUT, \&my_filter)
> 
>   die "err message\n"
>     unless mapscript.msIO_PushFilter(MS_STDOUT, \&my_filter)
> 
>   oldFilter = mapscript.msIO_PopFilter(MS_STDOUT)
> 
> This might get a little too tricky between C code and script code, so 
> your proposal might be best. Anyway, just sharing some ideas on this.

Exactly.  This sort of interlanguage calling gets quite tricky.  I've
done it in the past with hand crafted mechanisms for Python.  But in
general I'm afraid of it.  I'd also venture that io filtering isn't
the most compelling case for this mechanism in mapserver.  If we
were willing to "bite this bullet", I'd be more interested in using
it for user provided classification and styling procedures.

> Overall performance - I am glad to see your note about testing
> performance. If there is any significant impact to performance in the
> simple draw shapefile using cgi scenario, then you need to consider
> making this an optional build time configuration option. I support
> clients drawing 1/2 billion +- map draws a year and have pressure to cut
> the draw time in 1/2, so keep this in mind ;)

Understood.  If the USE_MAPIO macro is not defined, then the io will
revert to the old direct methods, and I anticipate retaining that as
an option.  My plan was that USE_MAPIO would be "on" by default but if
we find there is a substantial speed penalty in a default configuration
we may need to reconsider the default.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org