[mapguide-internals] RE: The pinned connection

Sun Oct 25 21:17:56 EDT 2009

I've been watching this thread for awhile now, it's all very interesting. At some point either Haris or UV asked for clarification around the use cases. I tend to agree with who ever made that comment. From a project perspective there has been enough concern raised over this RFC that I think we should re-submit the RFC for vote once we decide what the right thing is.

Now to the technical aspects, but first some history. The original design of the FeatureServices API dating back to 2005 or so actually looked like this:

conn = featureService.GetConnection(resId);
reader = conn.SelectFeatures(...);
reader.Close();
conn.Close();

At the time I discarded this design simply because I wanted a completely stateless interface for scalability reasons. In the end the reader had to be stateful and hence the connection state was persisted between the web and server tier anyway. However we still avoided having stateful connections held across HTTP requests. Holding connections across HTTP requests is a guaranteed scalability killer for a web application. I can recite tons of technologies that have tried to do this and failed.

So what's the use case and what's the scalability requirement of the application? E.g. exactly how many concurrent users do you anticipate will need to hold a connection over HTTP? 

Bob

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Sunday, October 25, 2009 8:00 AM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] RE: The pinned connection

I think we need to hear from Leaf and others who already worked on existing
RFCs. They need to feel comfortable with suggested changes.

Haris

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV
Sent: Sunday, October 25, 2009 2:18 PM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] RE: The pinned connection

good. so if no previous RFC78 API needs continuation thats absolutely fine.

Is it appropriate to create a new RFC with references to the previous 
RFC78 & RFC86?
I can do that if needed.

Haris Kurtagic wrote:
> +1
> Haris
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Trevor
> Wekel
> Sent: Sunday, October 25, 2009 10:59 AM
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] RE: The pinned connection
>
> I personally like
>
> conn = MgFeatureService.GetFeatureConnection(resourceId);
> conn.BeginTransaction();
> conn.CommitTransaction();
> conn.RollbackTransaction(); // I think rollback is a single word in
database
> lingo
>
>
> I did a quick check and many databases do not support nested transactions
so
> having a separate transaction object could lead to illegal use cases.
From
> what I can tell, having a single transaction per connection is more "SQL"
> like and should be easier to support for all database providers.
>
> I'm sure an Fdo expert will jump in here and correct me if I'm wrong...
>
>
> Thanks,
> Trevor
>
>   
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV
> Sent: October 24, 2009 5:59 AM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] RE: The pinned connection
>
> I have a proposal for further integration of RFC78 and RFC86.
> This just takes the one from Haris and adds a few more methods to stay 
> compatible.
>
>
> I like an additional transaction object. With overloading this would 
> support the original RFC78 as well (unless there is a SWIG limitation?)
> If MgTransaction is already implemented we can us it in the way below.
>  
> This is better for the same reason as exposing the dedicated connection 
> via the API as it makes the DB access code more explicit.
> This can be integrated like this and makes very clear and legible code:
>
> Full support also for renaming the connection to FeatureConnection or 
> DedicatedFeatureConnection which seems more descriptive english.
>
> MgFeatureConnection conn = nullptr;
> MgTransaction tran = nullptr;
>
> try
> {
>     conn = MgFeatureService.GetFeatureConnection(resourceId);
>     conn.BeginTransaction(); // <--- added
>     // or 
>   	  tran = conn.BeginTransaction(); // <--- minimal compatible change
> 			/// or 
> 		    tran = MgFeatureService.StartTransaction(resourceId);
> /// alternative from RFC78
>
>     MgFeatureService.ExecuteSql(resourceId, sql statement1, conn);
>     MgFeatureService.ExecuteSql(resourceId, sql statement2, conn); 
>     MgFeatureService.UpdateFeatures(resourceId,
> MgFeatureCommandCollection,conn);
>
> 			/// maybe the connection can be simply overloaded
> \/\/
> 			MgFeatureService.ExecuteSql(resourceId, sql
> statement1, tran); 
> 			MgFeatureService.ExecuteSql(resourceId, sql
> statement2, tran); 
> 			MgFeatureService.UpdateFeatures(resourceId,
> MgFeatureCommandCollection, tran);
>
> 			/// or if not overloaded (but then its not pretty
> anymore
> 			MgFeatureService.ExecuteSql(resourceId, sql
> statement1, null, tran); 
> 			MgFeatureService.ExecuteSql(resourceId, sql
> statement2, null, tran); 
> 			MgFeatureService.UpdateFeatures(resourceId,
> MgFeatureCommandCollection, null, tran);
>
>     conn.CommitTransaction();  // <--- added
> 		    	tran.Commit();	/// alternative
>
>     conn.Release();
> }
> catch(...)
> {
>     conn.RollBackTransaction();  // <---- added 
> 		    if(nullptr != tran) tran.Rollback();   /// alternative
>
> }
>
>
> Trevor Wekel wrote:
>   
>> I like this solution.  It associates the transaction with the connection
>>     
> and allows both the execute and the update to occur on the same connection
> with or without transaction support.  It effectively rolls 2 RFCs into
one.
>   
>> I would personally prefer MgFeatureConnection over MgPinnedConnection
>>     
> because the "Feature" connection is created from "Feature" service.
> MgConnection is ambiguous because MapGuide also opens TCP/IP connections
> from web tier to server.
>   
>> MgFeatureConnectionNotAvailable would be an appropriate throw from
>>     
> MgFeatureService.GetFeatureConnection()
>   
>> -----Original Message-----
>> From: mapguide-internals-bounces at lists.osgeo.org
>>     
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
>   
>> Sent: October 24, 2009 4:03 AM
>> To: 'MapGuide Internals Mail List'
>> Subject: RE: [mapguide-internals] RE: The pinned connection
>>
>> I was writing about just one solution ( I hope that my English doesn't
add
>> some misunderstanding :) ).
>>
>> As you wrote first example is what I was suggesting for first RFC68.
>> If functionality for RFC78 is done (or is in plan) I would extend
>>     
> connection
>   
>> so you would start transaction on connection.
>> Then we would not have two parameters to ExecuteSql but just one.
>> I will try to explain in code, will just add to your first example :
>>
>> MgPinnedConnection conn = nullptr;
>> try
>> {
>>     conn = MgFeatureService.GetPinnedConnection(resourceId);
>>     conn.BeginTransaction(); // <--- added
>>     	
>>     MgFeatureService.Update...
>>     MgFeatureService.Update...
>>     MgFeatureService.ExecuteSql(resourceId, sql statement1, conn);
>>     conn.CommitTransaction();  // <--- added
>>     MgFeatureService.ExecuteSql(resourceId, sql statement2, conn); 
>>     MgFeatureService.UpdateFeatures(resourceId,
>>     
> MgFeatureCommandCollection,
>   
>> conn);
>>     conn.Release();
>> }
>> catch (NoPinnedConnectionAvailableException ex)
>> {
>> .
>> }
>>
>>
>> And I would have suggestion to change the name from MgPinnedConnection to
>> MgConnection or MgFdoConnection or... To me this pinn.. sounds
unnecessary
>> it is just MG/FDO connection to underlying data source.
>>
>>
>> Haris
>>
>>  
>> -----Original Message-----
>> From: mapguide-internals-bounces at lists.osgeo.org
>> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Leaf Li
>> Sent: Saturday, October 24, 2009 2:40
>> To: mapguide-internals at lists.osgeo.org
>> Subject: [mapguide-internals] RE: The pinned connection
>>
>> Hi Haris,
>>
>> If my understanding is correct, you propose two solutions in our
>>     
> discussion.
>   
>> One is simpler. This solution will not affect MgTransaction.
>>  
>> MgPinnedConnection conn = nullptr;
>> try
>> {
>>     conn = MgFeatureService.GetPinnedConnection(resourceId);
>>
>>     MgFeatureService.ExecuteSql(resourceId, sql statement1, conn); 
>>     MgFeatureService.ExecuteSql(resourceId, sql statement2, conn); 
>>     MgFeatureService.UpdateFeatures(resourceId,
>>     
> MgFeatureCommandCollection,
>   
>> conn);
>>     conn.Release();
>> }
>> catch (NoPinnedConnectionAvailableException ex)
>> {
>> .
>> }
>>  
>>  
>> Another is in this way. It affects MgTransaction. So we have to change
>> current design of MapGuide RFC 78 - Enhance transaction capability to
>> Feature Service. For example, transaction isn't started, committed and
>> rollbacked by MgFeautreService. It is done by MgPinnedConnection.
>>  
>>  
>>   
>>     
>>> Conn1 = Open(FeatureSourceId)
>>> Execute SQL(Conn1,..)
>>> Open(Conn2)
>>> Execute SQL(Conn2,..)
>>> Open(Conn3)
>>> Conn3.StartTransaction()
>>> Conn3.Committ()
>>> Execute SQL(Conn3,..)
>>> Close(conn3)
>>> Close(Conn2)
>>> Close(Conn1)
>>>     
>>>       
>>  
>>  
>> For the second, it think it is the perfect way. I think you are talking
>> about the first solution in the email below. There are some issues for
the
>> first solution. One of issues is how to execute a transaction against a
>> existing pinned connection? Maybe we need to add the following method.
>> Right?
>>  
>> ExecuteSqlNonQuery(MgResourceIdentifier, string, MgTransaction,
>> MgPinnedConnection)
>>  
>> However, MgTransaction already occupies a connection when calling
>> FeatureService::BeginTransaction(.). How do we handle them in this time?
>>
>> So if we want to return a MgPinnedConnection object to users, we should
>>     
> use
>   
>> the second solution to start a transaction instead of
>> FeatureService::BeginTransaction(.).
>>
>> Thanks,
>> Leaf Li
>>
>>   
>>     
>>> Hi Li,
>>>
>>> I think that more work is needed for API as proposed in RFC because
there
>>>     
>>>       
>> is
>>   
>>     
>>> functionality around pinning and unpinning connections and keeping track
>>>       
> of
>   
>>> those. All functionality for the solution I would prefer is already
>>>       
> there,
>   
>>> it is just about the way how is it exposed.
>>>
>>> It does changes API but in same way as it changes with transaction
RFC78.
>>> My suggestion would be just to change that one and instead of adding
>>> transaction as new parameter add connection. And then on connection just
>>>     
>>>       
>> use
>>   
>>     
>>> transaction functionality which was developed for RFC78.
>>>     
>>>       
>> Haris_______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>   
>>     
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
>   

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals