[mapserver-dev] Proposal for GeoJSON support in MapServer
Dan Little
danlittle at yahoo.com
Thu Jan 29 09:35:41 EST 2009
"Set it up everytime" Just because code isn't a part of the core of Mapserver doesn't mean it isn't re-usable.
----- Original Message ----
> From: Bob Basques <Bob.Basques at ci.stpaul.mn.us>
> To: bfraser at geoanalytic.com
> Cc: mapserver-dev at lists.osgeo.org
> Sent: Wednesday, January 28, 2009 11:00:27 PM
> Subject: Re: [mapserver-dev] Proposal for GeoJSON support in MapServer
>
> Brent,
>
> Well, I have been accused of being ahead of the curve before. But it's always
> nice to hear it from new folks. :c)
>
> It's not so much to make MapServer be a plugin as it is to install a plug
> connector onto MapServer. Dan L's comments about lean and mean MapServer are
> what got me using it in the first place, and it's why I mentioned this idea in
> the first place, in lieu of adding directly to MapServer. I knew Dan L. was
> going to make some sort of remark as he did anyway. :c)
>
> As to the laziness remark. Yup, that's me, lazy. I don't think I should need
> to build the same thing around mapserver everytime I set up a new
> implementation, as in a cascaded one or one on a mobile device. If you do
> something often enough, isn't it a requirement (in the OpenSource Community) to
> automate the task? I really think there is something here that others could
> use, or I wouldn't have mentioned it.
>
> We can certainly build our own implementation (like the one we have already)
> into Mapserver, but isn't that going down the dilution road so to speak?
>
> bobb
>
>
>
> >>> Brent Fraser 01/28/09 4:50 PM >>>
> Ok, I think I get it (at least some of it). You want mapserver to act as a
> plugin (or a spatial data supplier/service) to non-OGC enabled desktop apps,
> preferably with an extendable set of methods (via templates?) to ask for data
> (spatial or attribute). And not necessarily though a web server; maybe
> explicitly linked in or dynamically loaded by the front-end app. Basically, a
> mapserver without the server.
>
> You're ahead of your time. I wasn't going to request this until 2010 (so I
> could build a mapping plugin to InkScape). ;)
>
> Brent Fraser
>
> Bob Basques wrote:
> > All,
> >
> > In case the previous message doesn't come up Stephens W's standards :c).
> >
> > Performance benefits:
> > * Being able to pre-request how many records will be returned for a
> particular mapview size, before actually retrieving them.
> > * Being able to request mutiple images (different layers)( via a single call,
> this would work good for all the multi-layer clients like GeoMoose and
> OpenLayers.
> > * Being able to request Multiple templated output formats from a single
> call. Point Queries, and ImageMap results for separate layers.
> >
> > Binding benefits:
> > * Binding a Calling structure based on application (I already have a AutoLISP
> connector for MapServer for feeding an AutoCAD session with Images and features)
> binding things like AutoCAD, MicroStation, Google Earth (Via KML), Database
> consumptio (Track requests for example to build out a new spatial layer for
> publication) would be of great benefit in our office.
> > * Binding other front ends to MapServer would help ease the distribution
> aspects. MapServer on CD or MapServer Mobile for example.
> > * Building out a MapServer Server/Client product to act as the bridge between
> the consumer and the supplier of data (kind like FME and how it uses a piping
> method to process things.)
> >
> > Again, I know this is all possible now already, but each item listed above
> requires some sort of extra chunk of something that needs to be added onto
> MapServer. I just want to be able to bind those chunks a little tighter is all.
> >
> > Cleare, or muddier?
> >
> > bobb
> >
> >
> >>>> Brent Fraser 01/28/09 10:49 AM >>>
> > Bob,
> >
> > Is your suggestion to make the output drivers dynamically loadable at run
> time (via .so/.dll modules), with a documented API so developers could supply
> drivers without disturbing the Mapserver code/doc? I like it (except for the
> user-support-nightmare aspect).
> >
> > Or is it something else? I don't understand the adding request modes
> thing. Do you have an example?
> >
> > Brent Fraser
> >
> >
> > Bob Basques wrote:
> >> All,
> >>
> >> I would like to suggest a slightly different approach to integration of this
> sort into MapServer. It will require some extra work though.
> >>
> >> A couple of years ago we were trying to do some specialized queries, mostly
> related to getCapabilties sort of requests. One thought was to build into
> MapServer a method for adding request modes. Where request parameters could be
> mapped to the mapserver internal calls. The idea would be to allow for
> compiling in a custom front end to mapserver.
> >>
> >> Our desire at the time (and its still present) was to be able to build a
> single executable for portability, with connectors to our custom clients. I
> know I'm not laying this out in a very well defined way as an idea (yet) but I
> hope some others can see the possible benefits of this type of approach.
> >>
> >> In the end, with a MODE parameter added into MapServer, it could start using
> Request plugin(s) frameworks that could be used to build all sorts of custom
> data transport operations. A connector front end to MapServer if you will.
> This leaves the option of compiling/binary building, still up to the
> implementor.
> >>
> >> This could easily go beyond just the GeoJson connector/outputs
> >>
> >> bobb
> >>
> >>
> >>
> >>>>> Jan Hartmann 01/28/09 8:54 AM >>>
> >> I have been thinking about the discussion we had about Canvas support in
> >> MapServer, and although the problem has been solved as far as I am
> >> concerned, perhaps a more general solution could be considered: to
> >> implement GeoJSON natively in MapServer, like it has been done in
> >> GeoServer. This would not only make Canvas support easier, but could
> >> have some further advantages , as you can see further on. So I would
> >> like to propose:
> >>
> >> 1) To implement a full GeoJSON representation of the complete MapServer
> >> map at the server side: layer geometries and styles. This would be an
> >> ASCII string and could be constructed quite easily, AFAICS, by looping
> >> throug the map elements and writing them to a string sourrounded by the
> >> appropriate square and curly brackets. This string could be made
> >> available to the calling client via a template variable, e.g. [GeoJSON].
> >> The only thing the client has to do is to declare a JavaScript header in
> >> the template file:
> >>
> >>
> >>
> >> and from that point on the complete internal representation of the map
> >> would be available (loopable and searcheable) in the variable gs as
> >> Javascript arrays and objects. Extra properties like bounding boxes for
> >> map and elements could be added to facilitate client side searching and
> >> zooming.
> >>
> >> Note that this GeoJSON template variable can be combined with a regular
> >> map returned as an image. The information in the GeoJSON template
> >> variable could be used e.g. to zoom to (parts of) the returned map.
> >>
> >> If you don't want the complete map tree, you can always use Steve's
> >> procedure as he described it below to return parts of the map via the
> >> TEMPLATE output driver. It's simple enough.
> >>
> >> 2) To add "mode=geojson" to the mapserver CGI mode control. This for
> >> people like me who prefer to work without templates (although they are a
> >> brilliant invention). The Javascript header would be in that case:
> >>
> >>
> >>
> >>
> >>
> >> 3) To add GeoJSON as output driver. This overlaps with 2), so it
> >> wouldn't be strictly necessary. The only difference would be that the
> >> javaScript call would have "mode=map"
> >>
> >> 4) To write separate generalized Javascript routines to convert the
> >> returned GeoJSON to Canvas and VRM. If the GeoJSON object will be
> >> implemented, I am willing to do this.
> >>
> >> 5) To implement the GeoJSON structures in the scripting languages. JSON
> >> can be converted to about any language available, so if a script
> >> language, e.g. PHP MapScript, can obtain the ASCII GeoJSON
> >> representation of the complete MapServer map, it can convert its members
> >> to native PHP arrays and objects. That would make the link between the
> >> MapServer engine and the scripting language engines much simpler and
> >> much more uniform. In the last resort, this would make redundant the
> >> SWIG interface, as the complete interface would now be implemented via
> >> an ASCII string.
> >>
> >> This could cause some performance overhead, but probably not very
> >> serious, especially since I would expect really performance-hungry
> >> applications to be written in MapServer CGI, while scripted applications
> >> will probably be more directed to analytic cartography and have other
> >> performance bottlenecks.
> >>
> >> 6) To implement new drivers via the GeoJSON interface. The Canvas driver
> >> is a simple example, but new drivers for other formats could just as
> >> well communicate with the MapServer core with GeoJSON, via a JSON-C
> >> interface. Perhaps even the GDAL/OGR bridge could benefit from this
> >> construction. However, in this case performance will be a major issue:
> >> it is not clear whether this way of writing driver will be more or less
> >> efficient than the plugin scenario discussed on this list.
> >>
> >> 7) To implement the MapFile in GeoJSON format. If the output of a
> >> MapServer map can be represented in GeoJSON, then so can the input
> >> MapFile. There has been for years a discussion on this list about a
> >> MapFile in SVG format. Perhaps a GeoJSON MapFile will be easier to
> >> implement, certainly if it can be derived from the template mechanism
> >> of 1). Once such a GeoJSON mapfile exists, it can easily be converted
> >> to SVG.
> >>
> >> Jan
> >>
> >>
> >> Jan Hartmann wrote:
> >>> As an afterthought: wouldn't it be an idea to output this template
> >>> directly via the OutputFormat, as a "GeoJSON" Driver type?. In that
> >>> case, MapServer would return the GeoJSON object with all dependent
> >>> properties directly as an ASCII string. If the calling HTML file had a
> >>> line like:
> >>>
> >>> > >>> src=http://mapserver.sara.nl/cgi-bin/mapserv?map=testGeoJSON.map&mode=map>
> >>>
> >>> the GeoJSON object would be loaded into the global namespace of the
> >>> web-page and could then be processed further. That way you wouldn't
> >>> need a template file, you wouldn't have to write separate templates
> >>> for each mapping application, and you would have a regular GeoJSON
> >>> driver, like GeoServer has. It wouldn't take much programming time
> >>> too, as far as I can see.
> >>>
> >>> Jan
> >>>
> >>> Jan Hartmann wrote:
> >>>> Wow, brilliant, this solves the problem. Just add a few JavaScript
> >>>> routines in the template to convert the GeoJSON object into Canvas
> >>>> commands and you are finished. You can process all elements after
> >>>> that via the GeoJSON arrays and never need any DOM tree (which isn't
> >>>> there anyway, as Christian Schmidt remarked).
> >>>>
> >>>> Just one question: how do you get styling information into the
> >>>> GeoJSON object? GeoJSON allows additional members at any level in a
> >>>> GeoJSON object, so these could be used to store colors, linewidths
> >>>> etc. Is it possible to get these parameters from the Layer's Class
> >>>> object into the template?
> >>>>
> >>>> Jan
> >>>>
> >>>> Steve Lime wrote:
> >>>>> MapServer can output GeoJSON via query calls to the CGI... Here's an
> example template:
> >>>>>
> >>>>> var csgSites = { "type": "FeatureCollection",
> >>>>> [resultset layer="sites"]"features": [
> >>>>> [feature limit=-1 trimlast=","]{"type": "Feature",
> >>>>> "geometry": {"type": "Point", "coordinates": [[shpxy]]},
> >>>>> "properties": {
> >>>>> "station": "[station]",
> >>>>> "long_name": "[long_name]",
> >>>>> "short_name": "[short_name]",
> >>>>> [item escape="none" name="storet_id" pattern="."
> format="'storet_id':'$value',"]
> >>>>> [item escape="none" name="usgs_id" pattern="."
> format="'usgs_id':'$value',"]
> >>>>> "has_telemetry": [has_telemetry],
> >>>>> "has_archive": [has_archive],
> >>>>> "has_water_chemistry": [has_water_chemistry],
> >>>>> "has_gagings": [has_gagings],
> >>>>> "has_photos": [has_photos]
> >>>>> }
> >>>>> },[/feature]
> >>>>> ][/resultset]
> >>>>> };
> >>>>>
> >>>>> You'd pair this with a new-style OUTPUTFORMAT like so:
> >>>>>
> >>>>> OUTPUTFORMAT
> >>>>> NAME json''
> >>>>> DRIVER 'TEMPLATE'
> >>>>> MIMETYPE 'aplication/json'
> >>>>> FORMATOPTION 'FILE=sites.js'
> >>>>> END
> >>>>>
> >>>>> Steve
> >>>>>
> >>>>>
> >>>>>>>> On 1/27/2009 at 10:25 AM, in message <497F358E.8000908 at uva.nl>, Jan
> Hartmann
> >>>>>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Another option would be to use GeoJSON. Is MapServer able to output
> >>>>>>
> >>>>>> GeoJSON, or is it going to be? I guess it would be much easier to write
> >>>>>> a conversion program from GeoJSON to Canvas than from SVG, and I am sure
> >>>>>> it would be much more efficient.
> >>>>>>
> >>>>>> Jan
> >>>>>>
> >>>>>> Stephen Woodbridge wrote:
> >>>>>>
> >>>>>>> Dan Little wrote:
> >>>>>>>
> >>>>>>>> I think there is some problems with the Canvas implementations across
> >>>>>>>> the different browsers. I've actually done quite a bit of work using
> >>>>>>>> the Canvas and the exCanvas extension from Google. While exCanvas
> >>>>>>>> SAYS it is a 100% implementation for IE there tends to be a number of
> >>>>>>>> quirks when it comes to actually placing objects in a drawing stack,
> >>>>>>>> animating them, or simply clearing the canvas at a predictable point.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Also, I think it would be much more to the benefit to simply have a
> >>>>>>>> VML output. A little bit of browser sniffing is easy to do with any
> >>>>>>>> JS application (or even some server side scripting) and you'd be
> >>>>>>>> outputting either SVG or VML in that browser's native preferred
> >>>>>>>> format. No goofy javascript library required.
> >>>>>>>>
> >>>>>>> I think converting some SVG into the proposed Canvas format would be a
> >>>>>>> good starting point so we can evaluate the performance. If consuming a
> >>>>>>> large Canvas file causes the browser to be slow or crash, then it
> >>>>>>> might not make sense to even consider moving forward with something
> >>>>>>> like this. It would seem to be a pretty straight forward task if we
> >>>>>>> can find someone that knows perl, Canvas and svg or is willing to do a
> >>>>>>> little research.
> >>>>>>>
> >>>>>>> -Steve W
> >>>>>>>
> >>>>>>>
> >>>>>>>> ----- Original Message ----
> >>>>>>>>
> >>>>>>>>> From: Brent Fraser To: Jan Hartmann
> >>>>>>>>> Cc: mapserver-dev at lists.osgeo.org Sent:
> >>>>>>>>> Tuesday, January 27, 2009 9:07:32 AM Subject: Re: [mapserver-dev]
> >>>>>>>>> Canvas support for MapServer
> >>>>>>>>>
> >>>>>>>>> Jan,
> >>>>>>>>>
> >>>>>>>>> So your thought is to have MapServer generate something like
> >>>>>>>>> (hacked from the example on
> >>>>>>>>> https://developer.mozilla.org/en/Canvas_tutorial/Basic_usage):
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Sure, it could be done. One way is to write another output driver
> >>>>>>>>> for Mapserver. Another is to use some server-side scripting to
> >>>>>>>>> ingest Mapserver's SVG and change into JavaScript-canvas code like
> >>>>>>>>> that above. A performance hit, but at least it would be on the
> >>>>>>>>> server side, and it might be a good way to prototype the Mapserver
> >>>>>>>>> driver.
> >>>>>>>>>
> >>>>>>>>> Brent
> >>>>>>>>>
> >>>>>>>>> Jan Hartmann wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Brent,
> >>>>>>>>>>
> >>>>>>>>>> First let me say that Canvas support is just an idea; I think it
> >>>>>>>>>> can be done
> >>>>>>>>>>
> >>>>>>>>> but whether it is advisable to do so is up to you to decide. I
> >>>>>>>>> liked the idea because the Canvas element is supported nowadays by
> >>>>>>>>> most browsers, unlike SVG which needs a plugin. It is also much
> >>>>>>>>> simpler, so it's easier to use for simple maps, and especially for
> >>>>>>>>> simple interactive operations like digitizing, or combining
> >>>>>>>>> server-side with client-side graphics. That's why I would prefer
> >>>>>>>>> Canvas above SVG, let alone the fact that I much prefer to program
> >>>>>>>>> in native Javascript above parsing XML. Most of my maps are simple
> >>>>>>>>> (I would guess that goes for many people here), so SVG is decidedly
> >>>>>>>>> overkill. The Canvas HTML element also fits nicely in the page
> >>>>>>>>> layout and can be processed easily by web-editors.
> >>>>>>>>>
> >>>>>>>>>> Your suggestion that Canvas could be implemented at the client
> >>>>>>>>>> side by
> >>>>>>>>>>
> >>>>>>>>> catching SVG output and transforming it into Canvas directives is
> >>>>>>>>> certainly viable. That is also Paul Spencer's view in his reply to
> >>>>>>>>> my email. Yet I would prefer a server-side approach: it's more
> >>>>>>>>> efficient (no second parsing of SVG code that had to be generated
> >>>>>>>>> in the first place), and I think the server-side code can be easily
> >>>>>>>>> adapted from the SVG driver. Canvas is a very simple format, just a
> >>>>>>>>> few graphic primitives, more like GD than SVG actually.
> >>>>>>>>>
> >>>>>>>>>> Again, I don't know how difficult it would be to add a Canvas
> >>>>>>>>>> outputformat,
> >>>>>>>>>>
> >>>>>>>>> and whether it fits at all into the MapServer design philosophy.
> >>>>>>>>> However, when it could be done, it would give me an a far more
> >>>>>>>>> easier tool than SVG to combine server-side and client-side
> >>>>>>>>> mapping.
> >>>>>>>>>
> >>>>>>>>>> Jan
> >>>>>>>>>>
> >>>>>>>>>> Brent Fraser wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Jan,
> >>>>>>>>>>>
> >>>>>>>>>>> My interest is in building a (or re-purposing an existing) tool
> >>>>>>>>>>> for
> >>>>>>>>>>>
> >>>>>>>>> Cartographic Layout, for printing or rendering a graphics format
> >>>>>>>>> for easy viewing (e.g. PDF). Steve's comments were about using
> >>>>>>>>> existing Web formats and syntax to describe the layout, and
> >>>>>>>>> existing(?) web editors to do the placement/moving of the layout
> >>>>>>>>> components.
> >>>>>>>>>
> >>>>>>>>>>> My (very limited) understanding of the HTML tag is that it's a
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>> "container" recognized by the browser, and vector (and raster)
> >>>>>>>>> graphics are rendered into it using client-side JavaScript. Sort
> >>>>>>>>> of the JavaScript equivalent of the SVG format.
> >>>>>>>>>
> >>>>>>>>>>> It would be possible to add another output format to Mapserver
> >>>>>>>>>>> to generate
> >>>>>>>>>>>
> >>>>>>>>> JavaScript code of canvas rendering methods, but a more elegant
> >>>>>>>>> approach might be to write a small(?) JavaScript module to ingest
> >>>>>>>>> Mapserver's SVG and render the objects to the canvas.
> >>>>>>>>>
> >>>>>>>>>>> Are there some specific features of the direct-canvas-output
> >>>>>>>>>>> approach not
> >>>>>>>>>>>
> >>>>>>>>> available in SVG that you need?
> >>>>>>>>>
> >>>>>>>>>>> Brent Fraser
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Jan Hartmann wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi folks,
> >>>>>>>>>>>>
> >>>>>>>>>>>> A few weeks ago I suggested in a thread about PDF support
> >>>>>>>>>>>> that Mapserver
> >>>>>>>>>>>>
> >>>>>>>>> could perhaps be made to support the new "Canvas" tag. There were
> >>>>>>>>> no reactions to this, so I don't know if this suggestion is
> >>>>>>>>> viable/advisable/practicable, or just downright stupid. Can anyone
> >>>>>>>>> comment? (from
> >>>>>>>>>
> http://lists.osgeo.org/pipermail/mapserver-dev/2009-January/008055.html)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>>>> I am wondering if MapServer support for the "Canvas" tag
> >>>>>>>>>>>>> could do what
> >>>>>>>>>>>>>
> >>>>>>>>> Steve suggests in a more simple way. Originally, only Safari, Opera
> >>>>>>>>> and Firefox supported this tag to allow (simple) 2D drawing, but
> >>>>>>>>> recently a surprisingly simple Javascript interface for IE has
> >>>>>>>>> become available, translating Canvas command to native IE VRML
> >>>>>>>>> commands. It requires just one single script tag in the web page.
> >>>>>>>>>
> >>>>>>>>>>>>> See the tutorial at the Mozilla site at:
> >>>>>>>>>>>>>
> >>>>>>>>> https://developer.mozilla.org/en/Canvas_tutorial
> >>>>>>>>>
> >>>>>>>>>>>>> and the IE interface (from the Google stables) at:
> >>>>>>>>>>>>>
> >>>>>>>>> http://excanvas.sourceforge.net/
> >>>>>>>>>
> >>>>>>>>>>>>> This is something I have long been looking for. The
> >>>>>>>>>>>>> graphics are very
> >>>>>>>>>>>>>
> >>>>>>>>> simple, so the functionality is nothing like PDF or SVG, but I
> >>>>>>>>> could imagine that a driver for MapServer could fulfill many needs.
> >>>>>>>>> I can use it already by letting Mapserver generate raw coordinates
> >>>>>>>>> and catching them with some sort of Ajax, but a separate driver
> >>>>>>>>> would be very neat of course.
> >>>>>>>>>
> >>>>>>>>>>>>> How about it?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Jan
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> ------------------------------------------------------------------------
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> _______________________________________________ mapserver-dev
> >>>>>>>>>>>> mailing list mapserver-dev at lists.osgeo.org
> >>>>>>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >>>>>>>>>>>>
> >>>>>>>>> _______________________________________________ mapserver-dev
> >>>>>>>>> mailing list mapserver-dev at lists.osgeo.org
> >>>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >>>>>>>>>
> >>>>>>>> _______________________________________________ mapserver-dev mailing
> >>>>>>>> list mapserver-dev at lists.osgeo.org
> >>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >>>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> mapserver-dev mailing list
> >>>>>> mapserver-dev at lists.osgeo.org
> >>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >>>>>>
> >>>>>
> >>>> ------------------------------------------------------------------------
> >>>>
> >>>> _______________________________________________
> >>>> mapserver-dev mailing list
> >>>> mapserver-dev at lists.osgeo.org
> >>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >>>>
> >> _______________________________________________
> >> mapserver-dev mailing list
> >> mapserver-dev at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >>
> > _______________________________________________
> > mapserver-dev mailing list
> > mapserver-dev at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >
> >
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
More information about the mapserver-dev
mailing list