[mapguide-internals] RE: MG Object stream serialization/deserialization

Wed Nov 4 22:20:06 EST 2009

Hi Haris,

Current:  It may be possible to write code in a latest version server to account for API differences.  However, I believe it would be prohibitively expensive to do so.

Future:  I think we would need to sit down and figure out a plan of attack for this.

One possible suggestion would be to add a version number as the first serialized parameter for every object serialized over TCP/IP.  This would allow us to handle minor API differences (ie. minor changes to objects but identical method signatures) in a straightforward way.  The objects could handle forward/backward compatibility themselves.  We would be forced to increment the global stream version if we did this.

For significant API changes, the Server would have to be specifically coded to handle both versions of the TCP/IP operation.  From what I recall, this should be possible with the existing infrastructure.  However, it may "pollute" the API over a period of time if we have to keep old C++ signatures around.  Instead, it might be possible and more straightforward to implement all of the version handling in the "Op" logic and force old TCP/IP signatures to use new C++ API methods.  Theoretically, ensuring exactly the same behaviour when using new C++ API methods for old TCP/IP signatures could be tricky if there are a lot of changes underneath the covers.

This would be an excellent topic for discussion at the PSC meeting tomorrow, at least as far as saying whether or not we actually want to pursue this.  As I mentioned before, the testing effort required to validate API compatibility could be painful.

The existing (open source) unit tests do not cover the API sufficiently to validate full compatibility.  I am sure internal Autodesk QA resources would have a better idea of the number of man years required to do something like this.  Developing a comprehensive test suite for 130 objects and 1400 method signatures seems like a multiple man year effort to me.

Thanks,
Trevor  

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: November 4, 2009 5:33 AM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] RE: MG Object stream serialization/deserialization

Current:
If I interpret correctly your email, you think there is no possibility to
implement application communicating to diff. version of MG using current
common libraries ?

Future:

Regarding versioning of serialization of objects I have experience with two
different large scale application (many objects). In one case versioning is
done "per stream" in another versioning it is "per object".

I prefer "per stream", which means that we use incrementally one version
number for any change on any object serialization/deserialization.
In current MG stream packet there is stream version which is unused.
Of 'course in order to work developers has to be carefully when changing
objects to increase global version number and add handling of previous
versions.

Regarding "default values" or handling previous versions, if there would be
such changes that it is impossible to came up with "default" value for
changed member we can still throw exception "invalid stream.." as it does
now.

Haris

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Trevor
Wekel
Sent: Wednesday, November 04, 2009 12:38 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: MG Object stream
serialization/deserialization

Hi Haris,

Yes.  That submission did add a member variable to MgClassDefinition in
order to support schema search related optimizations with some Fdo
providers.

Up to this point, the web extensions API has never supported backward
compatibility from a binary perspective.  You cannot successfully mix and
match different web extensions and servers.  As you have noticed, any change
in object signature breaks binary compatibility.

I believe it will be a huge amount of work to successfully code and test for
backward binary compatibility on an API this large.  If the PSC really wants
to go down this road, I may be willing to support the decision but that
means every contributor will be responsible for maintaining this with every
submission.  The permutations and combinations on the testing effort could
be painful to say the least.  It's hard enough to keep one API version
stable.  A mixed and matched the web extensions and server should probably
be considered a whole other API.  

Sincerely,
Trevor

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
Kurtagic
Sent: November 4, 2009 3:41 AM
To: 'MapGuide Internals Mail List'
Subject: [mapguide-internals] RE: MG Object stream
serialization/deserialization

Hi Trevor,

I believe I found when ClassDefinition serialization has changed.

Changeset 3905 <http://trac.osgeo.org/mapguide/changeset/3905>  for
trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp
<http://trac.osgeo.org/mapguide/browser/trunk/MgDev/Common/PlatformBase/Serv
ices/ClassDefinition.cpp?rev=3905> 

Timestammp

Author trevorwekel

Ticket  <http://trac.osgeo.org/mapguide/ticket/1002> #1002 Batch Processing
for Get Identity Properties

http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBase
/Services/ClassDefinition.cpp at 3905
<http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBas
e/Services/ClassDefinition.cpp at 3905&new=trunk/MgDev/Common/PlatformBase/Serv
ices/ClassDefinition.cpp at 3905>
&new=trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp at 3905

It feels bit strange that in previous versions schema name from class def.
was omitted ( but that is how 2.1 Beta and MG Ent 2010 works ).

This change was really for that ticket  "Batch Processing for Get Identity
Properties" or ?

I would like to be able to communicate with MG 2.0.2, MG 2.1 MG Ent. 2010
with one application using standard MG libraries. 

Any idea If that would be possible ?

Haris

From: Haris Kurtagic [mailto:haris at sl-king.com] 
Sent: Tuesday, November 03, 2009 11:47 PM
To: 'MapGuide Internals Mail List'
Subject: MG Object stream serialization/deserialization

Hi ,

I have lot of problems while trying to communicate with different versions
of MapGuide. I am developing application which should communicate with MG
using MG common libraries: MapGuideCommon, PlatformBase,...

Problem is that I can't find two version of MG which will work one against
other and work equally.

For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch
against current branch version. Also for example I can't use any of those
versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while
current 2.1 branch will throw exception InvalidStreamHeader.

At least one of problems is that ClassDefinition in one version is
serializing SchemaName in another it is not. So those versions can't be used
one against each other.

I have two questions:

1.  I really can't' find consistency in our svn branches and trunk how and
when is changing how MG objects are serialized/desterilized. I hope somebody
can light me up little bit when/ why are this changes done.

2. We don't have any versioning of serialization process of objects. I think
that could be issue ( hopefully one day we could imagine farm of MapGuide's
).

    Can we add versions of objects to serialization process ?

I would appreciate any help/ideas how to solve this mess, how to have one
application communicating with different MG versions.

Thank you,

Haris

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals