[mapguide-internals] MG Object stream serialization/deserialization

[Trevor Wekel]

Hey guys,

The programmatic (web extensions) API which is based on wrapped C++ objects is much more involved than the HTTP API.  As far as I know, there are about 100 methods exposed in the HTTP API.  The web extensions API consists of approximately 130 classes and 1400 method calls.  It is much richer and much more complicated.

The problem with backward compatibility is what should the API do if objects change.  If you add a new data member to a C++ class which gets passed around in the API, how do you deal with it?  Probably the only solution is to default the new data member to something "reasonable" if an old API signature is used.  However, this is avoided if the object definitions on the web extensions and the server are in sync.  That way you know what you are getting instead of some side effect of "somewhat" incompatible API versions and objects.

I agree, backward compatibility would make it easier from a deployment perspective but it is a non-trivial piece of work to maintain backward compatibility.  I'm sure Sun and Microsoft do a very good job of maintaining backward compatibility but MapGuide does not have hundreds of developers and testers working on the project.

If someone can get backward compatibility buy in from all the contributors, I may be willing to support such an initiative.  I believe this would mean that every RFC moving forward would have to explicitly consider and address coding and testing of any backward compatibility issues.

Trevor
  

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Kenneth Skovhede, GEOGRAF A/S
Sent: November 4, 2009 1:12 AM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] MG Object stream serialization/deserialization

I agree that the API should be fully backwards compatible.
It's a real pain that the client must know in advance what version of 
the API to use.
This is especially true for .Net which has a less-than-easy method for 
version selection on dll's.

The HTTP API is fully version capable, but the native API (aka WebAPI) 
is not.
The reason for this is that the native API is actually wrapping C++ classes.

Wrapping these classes reproduce some functionality from the MapGuide 
server on the
client (buffering, etc.).

Since these C++ classes are not clearly defined, they change between 
releases without notice,
breaking the API compatibility.

If there was a version field, the serializer could take care of this, 
and select a proper default
value for missing fields.

Unfortunately, there is no serializer, only custom code inside the 
classes that do this, making
it fairly difficult to maintain.

I belive that the root issue is that the API is too deeply nested in the 
server.
I would propose that the API sends binary versions of the same requests 
that the HTTP API
sends. It would then be a matter of implementing a light interface that 
resembles the current
http request parser, and reuses the same functions.

I hear speed concerns incoming already :)

The overhead for the HTTP API is significant, both because of the HTTP 
protocol itself, but also
due to the http parameter parsing and the xml input/output.

Both these issues could be resolved by using a binary format of the 
messages, and allowing the
server to accept these binary messages directly (like the WebAPI does now).

The binary format should not be a repetition of the current system, but 
rather
a generic format, something similar to "binary JSON", so the actual 
class layout
and message format are independant.

Such a seperation is also much safer, as it is easier to unittest the 
binary parser,
and the messages could be allowed over http, unlike the current WebAPI.

Regards, Kenneth Skovhede, GEOGRAF A/S



UV skrev:
> I very much support Haris ideas.
> Versioning of serialized objects as well as the API calls and DLL 
> versions is something that is state-of-the-art in the IT field. Those 
> pieces of extra information  help to work efficiently as it helps to 
> avoid a number frustrating build & deployment scenarios.
>
> A seperate external text document containing configuration information 
> tends to be outdated most of the time and arriving too late.
>
> One issue is the kind of implicit dependency on a certain FDO release.
> For the releases this piece of information is written down somewhere....
> however I dont understand why the DLL loaders dont check those 
> versions????
>
> The versioning of serialized objects is another related issue. THose 
> are approaches to write robuster code.
>
> A lot of such configuration data should be made part of the CI system.
> This is how those issues are addressed in successful IT projects as it 
> makes the dependencies
> a part of the build which shows up as a TAG in the repository.
>
> Leaving this as loose as it is in the moment causes pain and more pain 
> for too many people.
> I would call this a part of the evolutionary step from inhouse 
> development to open source!
> In a small local team such formalisms can be avoided by asking the 
> correct team member sitting near. However this does not scale well and 
> in respect to the community those issues should be addresses
> more thoroughly  at some point hopefully soon.
>
> UV
>
>
>
>
> Haris Kurtagic wrote:
>> Hi ,
>>
>>  
>>
>> I have lot of problems while trying to communicate with different 
>> versions
>> of MapGuide. I am developing application which should communicate 
>> with MG
>> using MG common libraries: MapGuideCommon, PlatformBase,...
>>
>>  
>>
>> Problem is that I can't find two version of MG which will work one 
>> against
>> other and work equally.
>>
>> For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch
>> against current branch version. Also for example I can't use any of 
>> those
>> versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 
>> 2010 while
>> current 2.1 branch will throw exception InvalidStreamHeader.
>>
>>  
>>
>> At least one of problems is that ClassDefinition in one version is
>> serializing SchemaName in another it is not. So those versions can't 
>> be used
>> one against each other.
>>
>>  
>>
>> I have two questions:
>>
>>  
>>
>> 1.  I really can't' find consistency in our svn branches and trunk 
>> how and
>> when is changing how MG objects are serialized/desterilized. I hope 
>> somebody
>> can light me up little bit when/ why are this changes done.
>>
>>  
>>
>> 2. We don't have any versioning of serialization process of objects. 
>> I think
>> that could be issue ( hopefully one day we could imagine farm of 
>> MapGuide's
>> ).
>>
>>     Can we add versions of objects to serialization process ?
>>
>>  
>>
>> I would appreciate any help/ideas how to solve this mess, how to have 
>> one
>> application communicating with different MG versions.
>>
>>  
>>
>> Thank you,
>>
>> Haris
>>
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>   
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals