[mapserver-dev] Proposal for GeoJSON support in MapServer
Brent Fraser
bfraser at geoanalytic.com
Wed Jan 28 11:48:42 EST 2009
Bob,
Is your suggestion to make the output drivers dynamically loadable at run time (via .so/.dll modules), with a documented API so developers could supply drivers without disturbing the Mapserver code/doc? I like it (except for the user-support-nightmare aspect).
Or is it something else? I don't understand the adding request modes thing. Do you have an example?
Brent Fraser
Bob Basques wrote:
> All,
>
> I would like to suggest a slightly different approach to integration of this sort into MapServer. It will require some extra work though.
>
> A couple of years ago we were trying to do some specialized queries, mostly related to getCapabilties sort of requests. One thought was to build into MapServer a method for adding request modes. Where request parameters could be mapped to the mapserver internal calls. The idea would be to allow for compiling in a custom front end to mapserver.
>
> Our desire at the time (and its still present) was to be able to build a single executable for portability, with connectors to our custom clients. I know I'm not laying this out in a very well defined way as an idea (yet) but I hope some others can see the possible benefits of this type of approach.
>
> In the end, with a MODE parameter added into MapServer, it could start using Request plugin(s) frameworks that could be used to build all sorts of custom data transport operations. A connector front end to MapServer if you will. This leaves the option of compiling/binary building, still up to the implementor.
>
> This could easily go beyond just the GeoJson connector/outputs
>
> bobb
>
>
>
>>>> Jan Hartmann <j.l.h.hartmann at uva.nl> 01/28/09 8:54 AM >>>
> I have been thinking about the discussion we had about Canvas support in
> MapServer, and although the problem has been solved as far as I am
> concerned, perhaps a more general solution could be considered: to
> implement GeoJSON natively in MapServer, like it has been done in
> GeoServer. This would not only make Canvas support easier, but could
> have some further advantages , as you can see further on. So I would
> like to propose:
>
> 1) To implement a full GeoJSON representation of the complete MapServer
> map at the server side: layer geometries and styles. This would be an
> ASCII string and could be constructed quite easily, AFAICS, by looping
> throug the map elements and writing them to a string sourrounded by the
> appropriate square and curly brackets. This string could be made
> available to the calling client via a template variable, e.g. [GeoJSON].
> The only thing the client has to do is to declare a JavaScript header in
> the template file:
>
> <!-- pt>
> var gs = eval([GeoJSON])
> </scr -->
>
> and from that point on the complete internal representation of the map
> would be available (loopable and searcheable) in the variable gs as
> Javascript arrays and objects. Extra properties like bounding boxes for
> map and elements could be added to facilitate client side searching and
> zooming.
>
> Note that this GeoJSON template variable can be combined with a regular
> map returned as an image. The information in the GeoJSON template
> variable could be used e.g. to zoom to (parts of) the returned map.
>
> If you don't want the complete map tree, you can always use Steve's
> procedure as he described it below to return parts of the map via the
> TEMPLATE output driver. It's simple enough.
>
> 2) To add "mode=geojson" to the mapserver CGI mode control. This for
> people like me who prefer to work without templates (although they are a
> brilliant invention). The Javascript header would be in that case:
>
> <!-- pt>
> var gs =
> </scr -->
> <!-- pt
> src=http://mapserver.sara.nl/cgi-bin-mapserv.map=GeoJSON.map&mode=geojson></scr -->
> <!-- pt>
> gs = eval(gs)
> </scr -->
>
> 3) To add GeoJSON as output driver. This overlaps with 2), so it
> wouldn't be strictly necessary. The only difference would be that the
> javaScript call would have "mode=map"
>
> 4) To write separate generalized Javascript routines to convert the
> returned GeoJSON to Canvas and VRM. If the GeoJSON object will be
> implemented, I am willing to do this.
>
> 5) To implement the GeoJSON structures in the scripting languages. JSON
> can be converted to about any language available, so if a script
> language, e.g. PHP MapScript, can obtain the ASCII GeoJSON
> representation of the complete MapServer map, it can convert its members
> to native PHP arrays and objects. That would make the link between the
> MapServer engine and the scripting language engines much simpler and
> much more uniform. In the last resort, this would make redundant the
> SWIG interface, as the complete interface would now be implemented via
> an ASCII string.
>
> This could cause some performance overhead, but probably not very
> serious, especially since I would expect really performance-hungry
> applications to be written in MapServer CGI, while scripted applications
> will probably be more directed to analytic cartography and have other
> performance bottlenecks.
>
> 6) To implement new drivers via the GeoJSON interface. The Canvas driver
> is a simple example, but new drivers for other formats could just as
> well communicate with the MapServer core with GeoJSON, via a JSON-C
> interface. Perhaps even the GDAL/OGR bridge could benefit from this
> construction. However, in this case performance will be a major issue:
> it is not clear whether this way of writing driver will be more or less
> efficient than the plugin scenario discussed on this list.
>
> 7) To implement the MapFile in GeoJSON format. If the output of a
> MapServer map can be represented in GeoJSON, then so can the input
> MapFile. There has been for years a discussion on this list about a
> MapFile in SVG format. Perhaps a GeoJSON MapFile will be easier to
> implement, certainly if it can be derived from the template mechanism
> of 1). Once such a GeoJSON mapfile exists, it can easily be converted
> to SVG.
>
> Jan
>
>
> Jan Hartmann wrote:
>> As an afterthought: wouldn't it be an idea to output this template
>> directly via the OutputFormat, as a "GeoJSON" Driver type?. In that
>> case, MapServer would return the GeoJSON object with all dependent
>> properties directly as an ASCII string. If the calling HTML file had a
>> line like:
>>
>> <!-- pt
>> src=http://mapserver.sara.nl/cgi-bin/mapserv?map=testGeoJSON.map&mode=map>
>>
>> the GeoJSON object would be loaded into the global namespace of the
>> web-page and could then be processed further. That way you wouldn't
>> need a template file, you wouldn't have to write separate templates
>> for each mapping application, and you would have a regular GeoJSON
>> driver, like GeoServer has. It wouldn't take much programming time
>> too, as far as I can see.
>>
>> Jan
>>
>> Jan Hartmann wrote:
>>> Wow, brilliant, this solves the problem. Just add a few JavaScript
>>> routines in the template to convert the GeoJSON object into Canvas
>>> commands and you are finished. You can process all elements after
>>> that via the GeoJSON arrays and never need any DOM tree (which isn't
>>> there anyway, as Christian Schmidt remarked).
>>>
>>> Just one question: how do you get styling information into the
>>> GeoJSON object? GeoJSON allows additional members at any level in a
>>> GeoJSON object, so these could be used to store colors, linewidths
>>> etc. Is it possible to get these parameters from the Layer's Class
>>> object into the template?
>>>
>>> Jan
>>>
>>> Steve Lime wrote:
>>>> MapServer can output GeoJSON via query calls to the CGI... Here's an example template:
>>>>
>>>> var csgSites = { "type": "FeatureCollection",
>>>> [resultset layer="sites"]"features": [
>>>> [feature limit=-1 trimlast=","]{"type": "Feature",
>>>> "geometry": {"type": "Point", "coordinates": [[shpxy]]},
>>>> "properties": {
>>>> "station": "[station]",
>>>> "long_name": "[long_name]",
>>>> "short_name": "[short_name]",
>>>> [item escape="none" name="storet_id" pattern="." format="'storet_id':'$value',"]
>>>> [item escape="none" name="usgs_id" pattern="." format="'usgs_id':'$value',"]
>>>> "has_telemetry": [has_telemetry],
>>>> "has_archive": [has_archive],
>>>> "has_water_chemistry": [has_water_chemistry],
>>>> "has_gagings": [has_gagings],
>>>> "has_photos": [has_photos]
>>>> }
>>>> },[/feature]
>>>> ][/resultset]
>>>> };
>>>>
>>>> You'd pair this with a new-style OUTPUTFORMAT like so:
>>>>
>>>> OUTPUTFORMAT
>>>> NAME json''
>>>> DRIVER 'TEMPLATE'
>>>> MIMETYPE 'aplication/json'
>>>> FORMATOPTION 'FILE=sites.js'
>>>> END
>>>>
>>>> Steve
>>>>
>>>>
>>>>>>> On 1/27/2009 at 10:25 AM, in message <497F358E.8000908 at uva.nl>, Jan Hartmann
>>>>>>>
>>>> <j.l.h.hartmann at uva.nl> wrote:
>>>>
>>>>> Another option would be to use GeoJSON. Is MapServer able to output
>>>>>
>>>>> GeoJSON, or is it going to be? I guess it would be much easier to write
>>>>> a conversion program from GeoJSON to Canvas than from SVG, and I am sure
>>>>> it would be much more efficient.
>>>>>
>>>>> Jan
>>>>>
>>>>> Stephen Woodbridge wrote:
>>>>>
>>>>>> Dan Little wrote:
>>>>>>
>>>>>>> I think there is some problems with the Canvas implementations across
>>>>>>> the different browsers. I've actually done quite a bit of work using
>>>>>>> the Canvas and the exCanvas extension from Google. While exCanvas
>>>>>>> SAYS it is a 100% implementation for IE there tends to be a number of
>>>>>>> quirks when it comes to actually placing objects in a drawing stack,
>>>>>>> animating them, or simply clearing the canvas at a predictable point.
>>>>>>>
>>>>>>>
>>>>>>> Also, I think it would be much more to the benefit to simply have a
>>>>>>> VML output. A little bit of browser sniffing is easy to do with any
>>>>>>> JS application (or even some server side scripting) and you'd be
>>>>>>> outputting either SVG or VML in that browser's native preferred
>>>>>>> format. No goofy javascript library required.
>>>>>>>
>>>>>> I think converting some SVG into the proposed Canvas format would be a
>>>>>> good starting point so we can evaluate the performance. If consuming a
>>>>>> large Canvas file causes the browser to be slow or crash, then it
>>>>>> might not make sense to even consider moving forward with something
>>>>>> like this. It would seem to be a pretty straight forward task if we
>>>>>> can find someone that knows perl, Canvas and svg or is willing to do a
>>>>>> little research.
>>>>>>
>>>>>> -Steve W
>>>>>>
>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>>
>>>>>>>> From: Brent Fraser <bfraser at geoanalytic.com> To: Jan Hartmann
>>>>>>>> <j.l.h.hartmann at uva.nl> Cc: mapserver-dev at lists.osgeo.org Sent:
>>>>>>>> Tuesday, January 27, 2009 9:07:32 AM Subject: Re: [mapserver-dev]
>>>>>>>> Canvas support for MapServer
>>>>>>>>
>>>>>>>> Jan,
>>>>>>>>
>>>>>>>> So your thought is to have MapServer generate something like
>>>>>>>> (hacked from the example on
>>>>>>>> https://developer.mozilla.org/en/Canvas_tutorial/Basic_usage):
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Sure, it could be done. One way is to write another output driver
>>>>>>>> for Mapserver. Another is to use some server-side scripting to
>>>>>>>> ingest Mapserver's SVG and change into JavaScript-canvas code like
>>>>>>>> that above. A performance hit, but at least it would be on the
>>>>>>>> server side, and it might be a good way to prototype the Mapserver
>>>>>>>> driver.
>>>>>>>>
>>>>>>>> Brent
>>>>>>>>
>>>>>>>> Jan Hartmann wrote:
>>>>>>>>
>>>>>>>>> Hi Brent,
>>>>>>>>>
>>>>>>>>> First let me say that Canvas support is just an idea; I think it
>>>>>>>>> can be done
>>>>>>>>>
>>>>>>>> but whether it is advisable to do so is up to you to decide. I
>>>>>>>> liked the idea because the Canvas element is supported nowadays by
>>>>>>>> most browsers, unlike SVG which needs a plugin. It is also much
>>>>>>>> simpler, so it's easier to use for simple maps, and especially for
>>>>>>>> simple interactive operations like digitizing, or combining
>>>>>>>> server-side with client-side graphics. That's why I would prefer
>>>>>>>> Canvas above SVG, let alone the fact that I much prefer to program
>>>>>>>> in native Javascript above parsing XML. Most of my maps are simple
>>>>>>>> (I would guess that goes for many people here), so SVG is decidedly
>>>>>>>> overkill. The Canvas HTML element also fits nicely in the page
>>>>>>>> layout and can be processed easily by web-editors.
>>>>>>>>
>>>>>>>>> Your suggestion that Canvas could be implemented at the client
>>>>>>>>> side by
>>>>>>>>>
>>>>>>>> catching SVG output and transforming it into Canvas directives is
>>>>>>>> certainly viable. That is also Paul Spencer's view in his reply to
>>>>>>>> my email. Yet I would prefer a server-side approach: it's more
>>>>>>>> efficient (no second parsing of SVG code that had to be generated
>>>>>>>> in the first place), and I think the server-side code can be easily
>>>>>>>> adapted from the SVG driver. Canvas is a very simple format, just a
>>>>>>>> few graphic primitives, more like GD than SVG actually.
>>>>>>>>
>>>>>>>>> Again, I don't know how difficult it would be to add a Canvas
>>>>>>>>> outputformat,
>>>>>>>>>
>>>>>>>> and whether it fits at all into the MapServer design philosophy.
>>>>>>>> However, when it could be done, it would give me an a far more
>>>>>>>> easier tool than SVG to combine server-side and client-side
>>>>>>>> mapping.
>>>>>>>>
>>>>>>>>> Jan
>>>>>>>>>
>>>>>>>>> Brent Fraser wrote:
>>>>>>>>>
>>>>>>>>>> Jan,
>>>>>>>>>>
>>>>>>>>>> My interest is in building a (or re-purposing an existing) tool
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>> Cartographic Layout, for printing or rendering a graphics format
>>>>>>>> for easy viewing (e.g. PDF). Steve's comments were about using
>>>>>>>> existing Web formats and syntax to describe the layout, and
>>>>>>>> existing(?) web editors to do the placement/moving of the layout
>>>>>>>> components.
>>>>>>>>
>>>>>>>>>> My (very limited) understanding of the HTML tag is that it's a
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> "container" recognized by the browser, and vector (and raster)
>>>>>>>> graphics are rendered into it using client-side JavaScript. Sort
>>>>>>>> of the JavaScript equivalent of the SVG format.
>>>>>>>>
>>>>>>>>>> It would be possible to add another output format to Mapserver
>>>>>>>>>> to generate
>>>>>>>>>>
>>>>>>>> JavaScript code of canvas rendering methods, but a more elegant
>>>>>>>> approach might be to write a small(?) JavaScript module to ingest
>>>>>>>> Mapserver's SVG and render the objects to the canvas.
>>>>>>>>
>>>>>>>>>> Are there some specific features of the direct-canvas-output
>>>>>>>>>> approach not
>>>>>>>>>>
>>>>>>>> available in SVG that you need?
>>>>>>>>
>>>>>>>>>> Brent Fraser
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Jan Hartmann wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi folks,
>>>>>>>>>>>
>>>>>>>>>>> A few weeks ago I suggested in a thread about PDF support
>>>>>>>>>>> that Mapserver
>>>>>>>>>>>
>>>>>>>> could perhaps be made to support the new "Canvas" tag. There were
>>>>>>>> no reactions to this, so I don't know if this suggestion is
>>>>>>>> viable/advisable/practicable, or just downright stupid. Can anyone
>>>>>>>> comment? (from
>>>>>>>> http://lists.osgeo.org/pipermail/mapserver-dev/2009-January/008055.html)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>>> I am wondering if MapServer support for the "Canvas" tag
>>>>>>>>>>>> could do what
>>>>>>>>>>>>
>>>>>>>> Steve suggests in a more simple way. Originally, only Safari, Opera
>>>>>>>> and Firefox supported this tag to allow (simple) 2D drawing, but
>>>>>>>> recently a surprisingly simple Javascript interface for IE has
>>>>>>>> become available, translating Canvas command to native IE VRML
>>>>>>>> commands. It requires just one single script tag in the web page.
>>>>>>>>
>>>>>>>>>>>> See the tutorial at the Mozilla site at:
>>>>>>>>>>>>
>>>>>>>> https://developer.mozilla.org/en/Canvas_tutorial
>>>>>>>>
>>>>>>>>>>>> and the IE interface (from the Google stables) at:
>>>>>>>>>>>>
>>>>>>>> http://excanvas.sourceforge.net/
>>>>>>>>
>>>>>>>>>>>> This is something I have long been looking for. The
>>>>>>>>>>>> graphics are very
>>>>>>>>>>>>
>>>>>>>> simple, so the functionality is nothing like PDF or SVG, but I
>>>>>>>> could imagine that a driver for MapServer could fulfill many needs.
>>>>>>>> I can use it already by letting Mapserver generate raw coordinates
>>>>>>>> and catching them with some sort of Ajax, but a separate driver
>>>>>>>> would be very neat of course.
>>>>>>>>
>>>>>>>>>>>> How about it?
>>>>>>>>>>>>
>>>>>>>>>>>> Jan
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________ mapserver-dev
>>>>>>>>>>> mailing list mapserver-dev at lists.osgeo.org
>>>>>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>>>>>>>
>>>>>>>> _______________________________________________ mapserver-dev
>>>>>>>> mailing list mapserver-dev at lists.osgeo.org
>>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>>>>
>>>>>>> _______________________________________________ mapserver-dev mailing
>>>>>>> list mapserver-dev at lists.osgeo.org
>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> mapserver-dev mailing list
>>>>> mapserver-dev at lists.osgeo.org
>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>>
>>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> mapserver-dev mailing list
>>> mapserver-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
More information about the mapserver-dev
mailing list