[mapserver-dev] Proposal for GeoJSON support in MapServer
Bob Basques
Bob.Basques at ci.stpaul.mn.us
Wed Jan 28 15:17:17 EST 2009
Brent,
Your're looking at it from the windows side (harder to implement) and I'm coming from the linux side(easier to implement :c)
But you've essential got the idea. I would think that this approach would make it quite easy to embed MapServer into client interfaces as well. My interest for example is in trying to set up a cascaded Mapservice that can be installed on a mobile device, with predefined integration aspects, as well as being able to call back to the mothership (Master MapServer instance) if needed to get more detailed info.
The aspect of installing some specialized connectors as a wrapper in front of MapServer is what I need to solve to make the cascading as simple to install as possible. Being able to bind the special calling methods to MapServer is the end product here. I've got some other applications for this as well, one being a AVL tracking system that is self contained.
The usual problems are in trying to set up the connector aspects to MapServer, both for templated output, as well as sending in pre-formatted requests, like GeoJon(ish) types of calls.
One last piece, is that this would seem to open the door for some interesting types of data reporting based on spatial queries, where an (almost) purely text based approach could be taken with regard to using MapServer as a DB interrogator.
All of this can be done already of course (we're doing it), but configuring a single service in multiple ways for product distribution has always seemed to end up being the short straw aspect of providing easy implementation strategies around MapServer.
I guess in short, I'm trying to make it possible to use MapServer as a desktop app of sorts, by combining with other Open Source stuff.
Anyway, just some more thoughts on the subject.
bobb
>>> Brent Fraser <bfraser at geoanalytic.com> 01/28/09 10:49 AM >>>
Bob,
Is your suggestion to make the output drivers dynamically loadable at run time (via .so/.dll modules), with a documented API so developers could supply drivers without disturbing the Mapserver code/doc? I like it (except for the user-support-nightmare aspect).
Or is it something else? I don't understand the adding request modes thing. Do you have an example?
Brent Fraser
Bob Basques wrote:
> All,
>
> I would like to suggest a slightly different approach to integration of this sort into MapServer. It will require some extra work though.
>
> A couple of years ago we were trying to do some specialized queries, mostly related to getCapabilties sort of requests. One thought was to build into MapServer a method for adding request modes. Where request parameters could be mapped to the mapserver internal calls. The idea would be to allow for compiling in a custom front end to mapserver.
>
> Our desire at the time (and its still present) was to be able to build a single executable for portability, with connectors to our custom clients. I know I'm not laying this out in a very well defined way as an idea (yet) but I hope some others can see the possible benefits of this type of approach.
>
> In the end, with a MODE parameter added into MapServer, it could start using Request plugin(s) frameworks that could be used to build all sorts of custom data transport operations. A connector front end to MapServer if you will. This leaves the option of compiling/binary building, still up to the implementor.
>
> This could easily go beyond just the GeoJson connector/outputs
>
> bobb
>
>
>
>>>> Jan Hartmann <j.l.h.hartmann at uva.nl> 01/28/09 8:54 AM >>>
> I have been thinking about the discussion we had about Canvas support in
> MapServer, and although the problem has been solved as far as I am
> concerned, perhaps a more general solution could be considered: to
> implement GeoJSON natively in MapServer, like it has been done in
> GeoServer. This would not only make Canvas support easier, but could
> have some further advantages , as you can see further on. So I would
> like to propose:
>
> 1) To implement a full GeoJSON representation of the complete MapServer
> map at the server side: layer geometries and styles. This would be an
> ASCII string and could be constructed quite easily, AFAICS, by looping
> throug the map elements and writing them to a string sourrounded by the
> appropriate square and curly brackets. This string could be made
> available to the calling client via a template variable, e.g. [GeoJSON].
> The only thing the client has to do is to declare a JavaScript header in
> the template file:
>
> <!-- pt>
> var gs = eval([GeoJSON])
> </scr -->
>
> and from that point on the complete internal representation of the map
> would be available (loopable and searcheable) in the variable gs as
> Javascript arrays and objects. Extra properties like bounding boxes for
> map and elements could be added to facilitate client side searching and
> zooming.
>
> Note that this GeoJSON template variable can be combined with a regular
> map returned as an image. The information in the GeoJSON template
> variable could be used e.g. to zoom to (parts of) the returned map.
>
> If you don't want the complete map tree, you can always use Steve's
> procedure as he described it below to return parts of the map via the
> TEMPLATE output driver. It's simple enough.
>
> 2) To add "mode=geojson" to the mapserver CGI mode control. This for
> people like me who prefer to work without templates (although they are a
> brilliant invention). The Javascript header would be in that case:
>
> <!-- pt>
> var gs =
> </scr -->
> <!-- pt
> src=http://mapserver.sara.nl/cgi-bin-mapserv.map=GeoJSON.map&mode=geojson></scr -->
> <!-- pt>
> gs = eval(gs)
> </scr -->
>
> 3) To add GeoJSON as output driver. This overlaps with 2), so it
> wouldn't be strictly necessary. The only difference would be that the
> javaScript call would have "mode=map"
>
> 4) To write separate generalized Javascript routines to convert the
> returned GeoJSON to Canvas and VRM. If the GeoJSON object will be
> implemented, I am willing to do this.
>
> 5) To implement the GeoJSON structures in the scripting languages. JSON
> can be converted to about any language available, so if a script
> language, e.g. PHP MapScript, can obtain the ASCII GeoJSON
> representation of the complete MapServer map, it can convert its members
> to native PHP arrays and objects. That would make the link between the
> MapServer engine and the scripting language engines much simpler and
> much more uniform. In the last resort, this would make redundant the
> SWIG interface, as the complete interface would now be implemented via
> an ASCII string.
>
> This could cause some performance overhead, but probably not very
> serious, especially since I would expect really performance-hungry
> applications to be written in MapServer CGI, while scripted applications
> will probably be more directed to analytic cartography and have other
> performance bottlenecks.
>
> 6) To implement new drivers via the GeoJSON interface. The Canvas driver
> is a simple example, but new drivers for other formats could just as
> well communicate with the MapServer core with GeoJSON, via a JSON-C
> interface. Perhaps even the GDAL/OGR bridge could benefit from this
> construction. However, in this case performance will be a major issue:
> it is not clear whether this way of writing driver will be more or less
> efficient than the plugin scenario discussed on this list.
>
> 7) To implement the MapFile in GeoJSON format. If the output of a
> MapServer map can be represented in GeoJSON, then so can the input
> MapFile. There has been for years a discussion on this list about a
> MapFile in SVG format. Perhaps a GeoJSON MapFile will be easier to
> implement, certainly if it can be derived from the template mechanism
> of 1). Once such a GeoJSON mapfile exists, it can easily be converted
> to SVG.
>
> Jan
>
>
> Jan Hartmann wrote:
>> As an afterthought: wouldn't it be an idea to output this template
>> directly via the OutputFormat, as a "GeoJSON" Driver type?. In that
>> case, MapServer would return the GeoJSON object with all dependent
>> properties directly as an ASCII string. If the calling HTML file had a
>> line like:
>>
>> <!-- pt
>> src=http://mapserver.sara.nl/cgi-bin/mapserv?map=testGeoJSON.map&mode=map>
>>
>> the GeoJSON object would be loaded into the global namespace of the
>> web-page and could then be processed further. That way you wouldn't
>> need a template file, you wouldn't have to write separate templates
>> for each mapping application, and you would have a regular GeoJSON
>> driver, like GeoServer has. It wouldn't take much programming time
>> too, as far as I can see.
>>
>> Jan
>>
>> Jan Hartmann wrote:
>>> Wow, brilliant, this solves the problem. Just add a few JavaScript
>>> routines in the template to convert the GeoJSON object into Canvas
>>> commands and you are finished. You can process all elements after
>>> that via the GeoJSON arrays and never need any DOM tree (which isn't
>>> there anyway, as Christian Schmidt remarked).
>>>
>>> Just one question: how do you get styling information into the
>>> GeoJSON object? GeoJSON allows additional members at any level in a
>>> GeoJSON object, so these could be used to store colors, linewidths
>>> etc. Is it possible to get these parameters from the Layer's Class
>>> object into the template?
>>>
>>> Jan
>>>
>>> Steve Lime wrote:
>>>> MapServer can output GeoJSON via query calls to the CGI... Here's an example template:
>>>>
>>>> var csgSites = { "type": "FeatureCollection",
>>>> [resultset layer="sites"]"features": [
>>>> [feature limit=-1 trimlast=","]{"type": "Feature",
>>>> "geometry": {"type": "Point", "coordinates": [[shpxy]]},
>>>> "properties": {
>>>> "station": "[station]",
>>>> "long_name": "[long_name]",
>>>> "short_name": "[short_name]",
>>>> [item escape="none" name="storet_id" pattern="." format="'storet_id':'$value',"]
>>>> [item escape="none" name="usgs_id" pattern="." format="'usgs_id':'$value',"]
>>>> "has_telemetry": [has_telemetry],
>>>> "has_archive": [has_archive],
>>>> "has_water_chemistry": [has_water_chemistry],
>>>> "has_gagings": [has_gagings],
>>>> "has_photos": [has_photos]
>>>> }
>>>> },[/feature]
>>>> ][/resultset]
>>>> };
>>>>
>>>> You'd pair this with a new-style OUTPUTFORMAT like so:
>>>>
>>>> OUTPUTFORMAT
>>>> NAME json''
>>>> DRIVER 'TEMPLATE'
>>>> MIMETYPE 'aplication/json'
>>>> FORMATOPTION 'FILE=sites.js'
>>>> END
>>>>
>>>> Steve
>>>>
>>>>
>>>>>>> On 1/27/2009 at 10:25 AM, in message <497F358E.8000908 at uva.nl>, Jan Hartmann
>>>>>>>
>>>> <j.l.h.hartmann at uva.nl> wrote:
>>>>
>>>>> Another option would be to use GeoJSON. Is MapServer able to output
>>>>>
>>>>> GeoJSON, or is it going to be? I guess it would be much easier to write
>>>>> a conversion program from GeoJSON to Canvas than from SVG, and I am sure
>>>>> it would be much more efficient.
>>>>>
>>>>> Jan
>>>>>
>>>>> Stephen Woodbridge wrote:
>>>>>
>>>>>> Dan Little wrote:
>>>>>>
>>>>>>> I think there is some problems with the Canvas implementations across
>>>>>>> the different browsers. I've actually done quite a bit of work using
>>>>>>> the Canvas and the exCanvas extension from Google. While exCanvas
>>>>>>> SAYS it is a 100% implementation for IE there tends to be a number of
>>>>>>> quirks when it comes to actually placing objects in a drawing stack,
>>>>>>> animating them, or simply clearing the canvas at a predictable point.
>>>>>>>
>>>>>>>
>>>>>>> Also, I think it would be much more to the benefit to simply have a
>>>>>>> VML output. A little bit of browser sniffing is easy to do with any
>>>>>>> JS application (or even some server side scripting) and you'd be
>>>>>>> outputting either SVG or VML in that browser's native preferred
>>>>>>> format. No goofy javascript library required.
>>>>>>>
>>>>>> I think converting some SVG into the proposed Canvas format would be a
>>>>>> good starting point so we can evaluate the performance. If consuming a
>>>>>> large Canvas file causes the browser to be slow or crash, then it
>>>>>> might not make sense to even consider moving forward with something
>>>>>> like this. It would seem to be a pretty straight forward task if we
>>>>>> can find someone that knows perl, Canvas and svg or is willing to do a
>>>>>> little research.
>>>>>>
>>>>>> -Steve W
>>>>>>
>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>>
>>>>>>>> From: Brent Fraser <bfraser at geoanalytic.com> To: Jan Hartmann
>>>>>>>> <j.l.h.hartmann at uva.nl> Cc: mapserver-dev at lists.osgeo.org Sent:
>>>>>>>> Tuesday, January 27, 2009 9:07:32 AM Subject: Re: [mapserver-dev]
>>>>>>>> Canvas support for MapServer
>>>>>>>>
>>>>>>>> Jan,
>>>>>>>>
>>>>>>>> So your thought is to have MapServer generate something like
>>>>>>>> (hacked from the example on
>>>>>>>> https://developer.mozilla.org/en/Canvas_tutorial/Basic_usage):
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Sure, it could be done. One way is to write another output driver
>>>>>>>> for Mapserver. Another is to use some server-side scripting to
>>>>>>>> ingest Mapserver's SVG and change into JavaScript-canvas code like
>>>>>>>> that above. A performance hit, but at least it would be on the
>>>>>>>> server side, and it might be a good way to prototype the Mapserver
>>>>>>>> driver.
>>>>>>>>
>>>>>>>> Brent
>>>>>>>>
>>>>>>>> Jan Hartmann wrote:
>>>>>>>>
>>>>>>>>> Hi Brent,
>>>>>>>>>
>>>>>>>>> First let me say that Canvas support is just an idea; I think it
>>>>>>>>> can be done
>>>>>>>>>
>>>>>>>> but whether it is advisable to do so is up to you to decide. I
>>>>>>>> liked the idea because the Canvas element is supported nowadays by
>>>>>>>> most browsers, unlike SVG which needs a plugin. It is also much
>>>>>>>> simpler, so it's easier to use for simple maps, and especially for
>>>>>>>> simple interactive operations like digitizing, or combining
>>>>>>>> server-side with client-side graphics. That's why I would prefer
>>>>>>>> Canvas above SVG, let alone the fact that I much prefer to program
>>>>>>>> in native Javascript above parsing XML. Most of my maps are simple
>>>>>>>> (I would guess that goes for many people here), so SVG is decidedly
>>>>>>>> overkill. The Canvas HTML element also fits nicely in the page
>>>>>>>> layout and can be processed easily by web-editors.
>>>>>>>>
>>>>>>>>> Your suggestion that Canvas could be implemented at the client
>>>>>>>>> side by
>>>>>>>>>
>>>>>>>> catching SVG output and transforming it into Canvas directives is
>>>>>>>> certainly viable. That is also Paul Spencer's view in his reply to
>>>>>>>> my email. Yet I would prefer a server-side approach: it's more
>>>>>>>> efficient (no second parsing of SVG code that had to be generated
>>>>>>>> in the first place), and I think the server-side code can be easily
>>>>>>>> adapted from the SVG driver. Canvas is a very simple format, just a
>>>>>>>> few graphic primitives, more like GD than SVG actually.
>>>>>>>>
>>>>>>>>> Again, I don't know how difficult it would be to add a Canvas
>>>>>>>>> outputformat,
>>>>>>>>>
>>>>>>>> and whether it fits at all into the MapServer design philosophy.
>>>>>>>> However, when it could be done, it would give me an a far more
>>>>>>>> easier tool than SVG to combine server-side and client-side
>>>>>>>> mapping.
>>>>>>>>
>>>>>>>>> Jan
>>>>>>>>>
>>>>>>>>> Brent Fraser wrote:
>>>>>>>>>
>>>>>>>>>> Jan,
>>>>>>>>>>
>>>>>>>>>> My interest is in building a (or re-purposing an existing) tool
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>> Cartographic Layout, for printing or rendering a graphics format
>>>>>>>> for easy viewing (e.g. PDF). Steve's comments were about using
>>>>>>>> existing Web formats and syntax to describe the layout, and
>>>>>>>> existing(?) web editors to do the placement/moving of the layout
>>>>>>>> components.
>>>>>>>>
>>>>>>>>>> My (very limited) understanding of the HTML tag is that it's a
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> "container" recognized by the browser, and vector (and raster)
>>>>>>>> graphics are rendered into it using client-side JavaScript. Sort
>>>>>>>> of the JavaScript equivalent of the SVG format.
>>>>>>>>
>>>>>>>>>> It would be possible to add another output format to Mapserver
>>>>>>>>>> to generate
>>>>>>>>>>
>>>>>>>> JavaScript code of canvas rendering methods, but a more elegant
>>>>>>>> approach might be to write a small(?) JavaScript module to ingest
>>>>>>>> Mapserver's SVG and render the objects to the canvas.
>>>>>>>>
>>>>>>>>>> Are there some specific features of the direct-canvas-output
>>>>>>>>>> approach not
>>>>>>>>>>
>>>>>>>> available in SVG that you need?
>>>>>>>>
>>>>>>>>>> Brent Fraser
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Jan Hartmann wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi folks,
>>>>>>>>>>>
>>>>>>>>>>> A few weeks ago I suggested in a thread about PDF support
>>>>>>>>>>> that Mapserver
>>>>>>>>>>>
>>>>>>>> could perhaps be made to support the new "Canvas" tag. There were
>>>>>>>> no reactions to this, so I don't know if this suggestion is
>>>>>>>> viable/advisable/practicable, or just downright stupid. Can anyone
>>>>>>>> comment? (from
>>>>>>>> http://lists.osgeo.org/pipermail/mapserver-dev/2009-January/008055.html)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>>> I am wondering if MapServer support for the "Canvas" tag
>>>>>>>>>>>> could do what
>>>>>>>>>>>>
>>>>>>>> Steve suggests in a more simple way. Originally, only Safari, Opera
>>>>>>>> and Firefox supported this tag to allow (simple) 2D drawing, but
>>>>>>>> recently a surprisingly simple Javascript interface for IE has
>>>>>>>> become available, translating Canvas command to native IE VRML
>>>>>>>> commands. It requires just one single script tag in the web page.
>>>>>>>>
>>>>>>>>>>>> See the tutorial at the Mozilla site at:
>>>>>>>>>>>>
>>>>>>>> https://developer.mozilla.org/en/Canvas_tutorial
>>>>>>>>
>>>>>>>>>>>> and the IE interface (from the Google stables) at:
>>>>>>>>>>>>
>>>>>>>> http://excanvas.sourceforge.net/
>>>>>>>>
>>>>>>>>>>>> This is something I have long been looking for. The
>>>>>>>>>>>> graphics are very
>>>>>>>>>>>>
>>>>>>>> simple, so the functionality is nothing like PDF or SVG, but I
>>>>>>>> could imagine that a driver for MapServer could fulfill many needs.
>>>>>>>> I can use it already by letting Mapserver generate raw coordinates
>>>>>>>> and catching them with some sort of Ajax, but a separate driver
>>>>>>>> would be very neat of course.
>>>>>>>>
>>>>>>>>>>>> How about it?
>>>>>>>>>>>>
>>>>>>>>>>>> Jan
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________ mapserver-dev
>>>>>>>>>>> mailing list mapserver-dev at lists.osgeo.org
>>>>>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>>>>>>>
>>>>>>>> _______________________________________________ mapserver-dev
>>>>>>>> mailing list mapserver-dev at lists.osgeo.org
>>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>>>>
>>>>>>> _______________________________________________ mapserver-dev mailing
>>>>>>> list mapserver-dev at lists.osgeo.org
>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> mapserver-dev mailing list
>>>>> mapserver-dev at lists.osgeo.org
>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>
>>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> mapserver-dev mailing list
>>> mapserver-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
More information about the mapserver-dev
mailing list