[mapguide-internals] Future Development Options for Web Server Extensions and new High Performance Viewer

Wed Dec 23 15:17:37 EST 2009

I share the opinion that the way to make things perform better without dealing with major amounts of middle tier pain is to push out more stuff to the client. Specifically, I think that a good AJAX/REST API, combined with client side rendering and logic would improve scalability significantly. Such API would not be a direct wrapper for everything MapGuide does. For example, the vast majority of queries a MapGuide process does are for rendering and they are performed against what is essentially constant feature data -- such queries can be pre-computed and cached as files, so the MapGuide server and web tier would not need to be involved at all -- Apache would just serve the pre-computed file. Only complex queries like searches or fetching more attribute data for selected items would necessitate requests to the server process.

I don't think that writing a dedicated client side browser plug-in, like 6.5, would be necessary. Modern browsers with HTML 5 support are more than powerful enough to render a 2D map using Javascript entirely. As an example (which will not be new to some people on this list), I have prototyped a MapGuide-like client-side map viewer which uses entirely HTML5 rendering and REST. You can check it out here (using Safari, Firefox or Chrome only):

http://13-9.com/c2 (Use the +/- keys or mouse wheel to zoom and arrow keys or middle-button-drag to pan, left-click to select).

Note the instant roll-over highlighting support and also the availability of a basic property window, which works significantly faster than MapGuide. Also note that all queries are cached in your browser, so as you pan away and then back to the same place, the data is not requested again.

So, given that it is possible to render with reasonable quality entirely on the client and also keep the client state there, it should be possible to make some improvements to data caching (so that the MapGuide server process is bypassed for constant data), and add some AJAX and REST APIs to the web tier which would make the need for PHP/Java/.Net wrappers obsolete -- in fact I think the whole web extensions can be reduced to stateless request translators that forward the translated requests to the server process, thus removing the need for wrappers and middle tier code completely. Of course, this is nothing new -- OpenLayers does that already, but I think it is possible to do far more than that in terms of interactivity, vector data and client side state keeping.

 Traian

________________________________________
From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Trevor Wekel [trevor_wekel at otxsystems.com]
Sent: Wednesday, December 23, 2009 1:46 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] Future Development Options for Web Server     Extensions and new High Performance Viewer

Hi everyone,

Carl and I started a conversation about native APIs and a high performance viewer last week.  I am glad he has brought it up on the internals list.  I believe there is merit in the discussion.

For native APIs, SWIG does give us a 3 APIs for one deal.  I believe this is more maintainable than implementing full native APIs in each language.  However, SWIG does generate very ugly C glue logic.  It might be worthwhile to consider development of a new tool that would go direct from C#/Java/PHP to C++.  This would make mixed mode debugging a little easier.  In addition, it may possible to have the new tool generate the serialization and deserialization logic for the majority of the API objects.  The autogeneration could also be enhanced to provide some level of API compatibility between current and future versions of MapGuide.  A prototype would have to be undertaken to see if this was feasible.

The high performance viewer would be a nice replacement for the deprecated DWF viewer.  If it is written in C++, some refactoring of the existing code base could yield a client side rendering engine with identical capabilities to the MapGuide server.  The refactored code base could be used as a basis for browser plugins and an installable viewer.

Moving all the rendering client side will improve performance.  Once the rendering is client side, client side selection could also be implemented.  Future work may include client side analysis capabilities using the existing Geometry API.  And a client side implementation of the resource and feature services may lead to disconnected viewer capability.

All of this would significantly reduce the load on the MapGuide Server, leading to better performance and scalability.  As we all know, the stylization and rendering capabilities of MapGuide Open Source blow MapGuide 6.5 out of the water.  But higher quality rendering does use more computing resources.

I agree C++ is more difficult to deal with than C#,Java, or PHP.  But we already have a very rich C++ code base which works on both Windows and Linux.

Thanks,
Trevor

________________________________________
From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On bBehalf Of Samuel Berger [samuel.berger at icfm.ch]
Sent: December 23, 2009 8:22 AM
To: 'MapGuide Internals Mail List'
Subject: AW: [mapguide-internals] Future Development Options for Web Server     Extensions and new High Performance Viewer

Some more thoughts

Also I worked extensively the last 3 years migrating from 6.5 to MGOS,
underestimating the huge difference...

My personal opinion:
the Web Server Extension (and even Fusion) is good for starting to have
quick results.
But the richer you want your app, sooner or later you will develop your own
client, suited for your business, communicating directly with the Web API.
We do this with Extjs library and OpenLayers. Performance is ok, if 1 person
is using it :)
The problem is, that for every zoom/pan the whole map rendering process
starts from zero. So if we will do it with a Silverlight client, we would
cache things in client, to be able to zoom and pan in a certain degree
without having to load the whole map...

I think the future is not in a rich Web Extension, but in a rich Web API.

As we have performance issues, I was looking for load balancing and XMLDB
synchronizing. I could find some discussion, but I don’t know what the
current state is.
Could somebody give a hint?

sam

-----Ursprüngliche Nachricht-----
Von: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] Im Auftrag von Zac
Spitzer
Gesendet: Mittwoch, 23. Dezember 2009 15:23
An: MapGuide Internals Mail List
Betreff: Re: [mapguide-internals] Future Development Options for Web Server
Extensions and new High Performance Viewer

a few thoughts

aren't stability & performance are more core server issues than the
web front end stuff

The future of the web is Javascript, rather than the whole re-invent the
wheel
approach, why not flesh out the ajax api. I shouldn't need to write
php/.net/java
server side code to be able to render a simple map.

Aren't most of the issues with MG a lot of the time FDO related? dead
& unclosed connections

Fixing the basic issues with SDF and SHP providers which cause us to
ship with connection
pooling disabled in the standard config, it a performance killer

Is it possible to abort requests if the http connnection is dropped?
Zooming in on
a slow map can DOS a mapguide server so easily

Being able to simply shutdown the server without waiting for things to
finish would
really make a huge difference as well.

Fix core issues before tackling the other stuff

2009/12/24 Jackie Ng <jumpinjackie at gmail.com>:
>
> Your ideas sound great in theory, but it how they will be applied in
practice
> that will be the real deciding issue.
>
> My main concern is the effort involved in having to develop and *maintain*
> the Web Server Extensions for <language of your choice> "by hand". Take
the
> effort required to develop one WSE in .net and multiply it by 3 (for PHP
and
> Java, assuming we still want to support these 3 platforms).
>
> There is also the high possibility of implementation-specific bugs and
> issues cropping up. At least SWIG sorts of provides a level playing field
in
> this respect (memory leaks aside, all 3 language bindings are
> logically/functionally equivalent).
>
> The developer effort to do this would be monumental. It just sounds
> infeasible from a resourcing standpoint. I doubt you'll see much
investment
> from Autodesk on such a developer effort. Currently I can only see this
> working as an "out-of-band" project.
>
> And all this for (what I presume to be) a high-performance viewer? Could
we
> not look at this from another angle?
>
> How about extending the mapagent http interface to support the
capabilities
> required by a high-performance client viewer? Or is the mapagent (or doing
> things over http) just not performant enough?
>
> On another (slightly unrelated) note, one of the "unique" uses I've seen
> with MapGuide 6.5 was the ability to create "fat" static MWF files such
that
> the MapGuide 6.5 ActiveX viewer can function as (or part of) an offline
> disconnected client viewer application.
>
> Would your proposed ideas enable such a use case in MapGuide Enterprise /
> Open Source?
>
> - Jackie
>
>
> carlj wrote:
>>
>> Having worked quite extensively with MapGuide for the past 2 years I was
>> hoping to get a discussion going regarding trying to address some of the
>> pain points of MapGuide Enterprise / Open Source.
>>
>> Many organisations have had difficulty migrating from MapGuide 6.5 to
>> enterprise due to difficulties with stability and performance. Rather
than
>> pointing fingers I hope to have a professional discussion of the options
>> for addressing some of the issues.
>>
>> One pain point is the MapGuide web server extensions. These are
>> implemented in native C++ and access to the development platforms such as
>> .Net, Java and PHP. This is done using SWIG to auto-generate objects in
>> the platforms using them.
>>
>> I want to propose that the use of SWIG should be gradually phased out in
>> favour of implementing the Web Server extensions in as pure .Net and Java
>> etc. I had originally discussed trying to port the web server extensions
>> in their entirety in one go but due to the volume of work involved this
>> could take a long time. This may still be a long term goal but in the
>> shorter term converting such things as the value objects and the core
>> communication / serialization components would be a good starting point.
>>
>> There is likely to be resistance to the suggestion of porting as there is
>> the disadvantage of having to duplicate changes across multiple
>> implementations. I personally believe that in the long run moving towards
>> the webserver extensions being implemented in the specific platform of
the
>> website will beneficial in the future. SWIG is known to leak memory if
not
>> used very carefully. Also many developers encounter lot of pain due to
not
>> being able to observe the internal state of the objects with which their
>> applications interact. Dealing with the library dependencies also causes
>> pain.
>>
>> As the Web Server Extensions communicate with the server via TCP/IP then
>> this seems to me to be a natural application boundary between the
MapGuide
>> server and the Web Server Extensions. All the platforms currently
>> supported by the web server extensions are able to communicate using
>> TCP/IP.
>>
>> There is an added bonus to having Web Server Extensions code ported to
>> Java and .Net and this is that it could be used as the basis of a new
>> plug-in based MapGuide viewer to fill the hole left by the unsupported
DWF
>> viewer. The new viewer could be higher performance than even the DWF
>> viewer was able to be because rather than rendering DWF (meaning the
>> server has to first package up data as DWF) a new viewer would instead be
>> able to use the ported Web Server Extensions to directly de-serialize the
>> C++ objects into objects in Java or .Net. These could then be used to
>> render the map purely on the client. With all the rendering pushed to the
>> client, the server has much less load and the overall solution should
>> scale better. As an added bonus these technologies also offer the
>> possibility of, for the first time, rendering 3D mapping data in the
>> browser on the client.
>>
>> For situations where this viewer would not work the AJAX viewer is still
>> going to be available. I do not believe that the AJAX viewer could ever
be
>> expected to achieve the performance possible with a plug-in based viewer.
>>
>> I have already had a discussion about this idea off-line but the
>> suggestion was to create another C++ based browser plug-in. I am not in
>> favour of using C++ for a browser plug-in as this would likely mean going
>> back to supporting one browser on one operating system which goes against
>> why an organisation would be pushing an application into the web. When
>> using native C++ on the client then it might as well be a full client
>> application rather than a web based one. Granted the Viewer API allows
>> some customization via JavaScript but I would much rather use a plug-in
>> technology which is cross platform and cross platform. I would favour
this
>> even if the performance were somewhat less than possible in native C++.
>>
>> There are more points that I could make but I think I would leave it
there
>> for starting a discussion.
>>
>>
>
> --
> View this message in context:
http://n2.nabble.com/Future-Development-Options-for-Web-Server-Extensions-an
d-new-High-Performance-Viewer-tp4208470p4208712.html
> Sent from the MapGuide Internals mailing list archive at Nabble.com.
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>

--
Zac Spitzer
Solution Architect / Director
Ennoble Consultancy Australia
http://www.ennoble.com.au
http://zacster.blogspot.com
+61 405 847 168
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals