[mapguide-internals] The pinned connection

[UV]

PROBLEM:
The dedicated connection API (here called pinned for some reason) is an 
implicit change of state of the server.
This "implicit nature" permits it to add this API transparently for 
other applications but it is the wrong approach passing API boundaries.

RISKS:
Loss of control.
Once the pinned connection has been requested further code uses it in 
the same way as a pooled connection.
This keeps the changes to the API small, but its a hack as it  changes 
server behaviour unvisible to the user!
In a larger application this is hard or impossible to control and will 
cause problems!

SOLUTION:
So introducing the API object for the "dedicated connection" is the 
right way to go.
For the previous API the FDO connection will still be a pooled one (we 
can still introduce configurable connection limits per user)

As Haris and me stated before: RFC78 and RFC86 need to be integrated 
properly, they are too closely connected.

So we could have a dedicated connection as an API object which also 
provides access to a gettransaction method.

WORKAROUND:
We could change the transaction semantics a little bit to allow the 
dedicated connection to persists after a commit!
like:

{
   MgTransaction tran = nullptr;

   tran = MgFeatureService.StartTransaction(resourceId);
   tran.Commit();
   /// connection is returned to pool only in  DTOR ~MgTransaction() 

   /// the following SQL runs in the same dedicated connection
   MgFeatureService.ExecuteSql(resourceId, sql statement1, tran);
   MgFeatureService.ExecuteSql(resourceId, sql statement2, tran); 
   MgFeatureService.UpdateFeatures(resourceId, MgFeatureCommandCollection, tran);

} /// end of context calls ~MgTransaction() end releases the dedicated connection


This seems to be able to deliver the required behavior without changing 
API!
However, creating the dedicated connection and getting the Transaction 
from it via API (Haris suggestions)
looks a lot better to me.

A discussion within a (preferably telephone) PSC meeting including 
interested contributors seems like the efficient way to resolve such issues.
Maybe there are other more efficient ways to discuss those things than a 
mailing list.....

I am happy to contribute some pieces here once decisions have been made.




Trevor Wekel wrote:
> I originally voted +1 on RFC 86.  However, the ongoing discussion on the internals list has brought up some interesting points.  Basically, we want to use the same Fdo connection across a set of Feature Service calls.  Pinning the connection as per RFC 86 makes this possible with minimal API change.
>
> However, it does conceal the fact that we are really using the same Fdo connection.  ADO.NET and other "web" languages expose database connections as API objects http://msdn.microsoft.com/en-us/library/ms254507.aspx.  To make the MapGuide API less obtuse, it would make sense to follow the same convention.
>
> We could create a new class MgFeatureConnection which would mirror the required execute, select, etc.  logic needed to meet the RFC 86 use cases.  We would also have to add an OpenFeatureConnection method to MgFeatureService.
>
> There would be no change to existing API functionality.  MgFeatureService would still use pooled connections.  In my opinion, "pinning" a database connection is a bit strange.  Just grabbing an API object that represents the connection is much easier to understand and more in line with (every other?) API implementation.
>
> Due to time constraints, I fully understand that implementing a pinned connection is a lot easier to implement than a whole new class.  However, it may be possible to leverage much of the existing Server Feature Service  implementation if an MgServerFeatureConnection parameter is added to various "command" classes.  If the connection is null, then the resource identifier would be used to grab a pooled connection.
>
> My +1 vote for RFC 86 still stands but I believe that a new API object would be a cleaner implementation.
>
>
> Thanks,
> Trevor
>
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
> Sent: October 20, 2009 11:52 AM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> I wouldn't call it deployment issue. The workaround you suggested is
> possible but very ugly.
>
> In my opinion API as it is suggested looks very unnatural, has limitations
> and is much different than most of standard APIs.
>
> I can guess that primary reason for such suggested solution in RFC was that
> changes in existing API are minimal but it makes API looks really strange.
>
> I would much rather see another set of ExecuteSql (e.g. ExecuteSql2 )
> commands then trying to combine old and new in such way.
>
> Haris
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Tuesday, October 20, 2009 19:41
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] The pinned connection
>
> You mean I cannot create multiple feature source IDs to the same data? That
> sounds like a deployment issue; if an application requires the use of a
> pinned connection and a regular connection at the same time, then I would
> provision 2 feature source ids to the same datastore. Or may be I don't
> understand that part?
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
> Sent: Tuesday, October 20, 2009 1:34 PM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> ?? Now you are getting me confused :)
>
> Isn't proposed solution in RFC "PinConnection(MgResourceIdentifier
> featureSourceId)" referring to Feature Source which identifies particular
> Feature Source and it is one unique id for one feature source ?
>
> Haris
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Tuesday, October 20, 2009 19:29
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] The pinned connection
>
> Why not? I just create 2 FSIDs and pinned them both. An application should
> be able to use any number of unique FSIDs.
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
> Sent: Tuesday, October 20, 2009 1:24 PM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> Yes, and one of issues of RFC is that you can't have another one
>
> Haris
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Tuesday, October 20, 2009 19:11
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] The pinned connection
>
> I don't think there is anything assumed about what connection is pinned. If
> a PinConnection is called using a specific feature source id, then all the
> calls using that feature source id will use the pinned connection associated
> with that fsid. There is no assumption made, the use of the pinned
> connection is explicit through the use of the feature source id.
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
> Sent: Tuesday, October 20, 2009 12:57 PM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> Hm, that is not true. Obviously it is a problem otherwise we wouldn't have
> this RFC.
> Connection pooling stays and it is common in web development but have
> nothing to do with this discussion and RFC. Regarding connection pooling
> nothing changes
>
> I understand what you want to do, I am suggesting to instead of just pin one
> connection and then assume in all other executions that user wanted to use
> that only pinned connection, return that connection ( in a way of new class
> or id or ... ) and then allow connections to be used in commands.
>
> Haris
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Tuesday, October 20, 2009 16:28
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] The pinned connection
>
> In the web environment not having control over which connection to use is
> NOT a problem; It's a feature. The connection pooling/sharing is a
> scalability feature. However, that can be an issue/restriction for some
> special/edge and uncommon cases. For those cases, we have a special solution
> that can be used; temporary pin the connection. BTW. This is similar to
> pinning a .Net managed object when a unmanaged object need to keep a
> reference to the managed object.
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
> Sent: Tuesday, October 20, 2009 10:11 AM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> Difference is that using feature source id introduce problems we have now
> and which we are trying to solve. And problem is as stated in RFC that
> developer doesn't have control over which database connection is used when
> he wants to execute commands.
> If instead of feature source id, parameter would be connection then we
> wouldn't have this issues and RFC.
>
> Equivalent changes to API are already suggested in transaction RFC78. So
> changes shouldn't be so difficult.
> Just instead of transaction as suggested in that RFC we could use connection
> as parameter.
>
> I understand that making such changes will change existing API, I would much
> reader introduce new clear API then changing existing one to support one
> particular case and create existing API more difficult to understand and
> use.
>
>
> This is how I believe API should look like:
>
> Conn1 = Open(FeatureSourceId)
> Execute SQL(Conn1,..)
> Open(Conn2)
> Execute SQL(Conn2,..)
> Open(Conn3)
> Conn3.StartTransaction()
> Conn3.Committ()
> Execute SQL(Conn3,..)
> Close(conn3)
> Close(Conn2)
> Close(Conn1)
>
>
> Haris
>
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Tuesday, October 20, 2009 15:30
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] The pinned connection
>
> I'm not sure what is the difference between using the proposed fdo
> connection and using the existing feature source id. It seems that
> introducing the new connection will require more changes to the API since
> all the existing commands use feature source id. If the proposed connection
> will replace the feature source id, then that may not be acceptable since
> that means clients will be pinning all connections. If this is an addition
> to the API where both feature source id and connection will be used, then
> this is cumbersome. I think pinning a connection is an exception/edge case
> that should not be used unless it's absolutely necessary. As a result an
> explicit API as suggested by the RFC is appropriate in this case.
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
> Sent: Tuesday, October 20, 2009 7:08 AM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> I don't think we confused transaction with what is called pinned connection.
> I think unnecessary confusion and limitations are in application when pinned
> connection is hidden (assumed) to application as it is suggested in RFC.
>
> My suggestion is that developer can acquire FDO connection and then use that
> connection in ExecuteSql... functions. In that way developer have clear
> picture what is he doing with which FDO connection and can execute any
> combination of operations.
>
> Reason why I am mentioning RFC78 is because same solution ( but only for
> transactions ) was introduced in the RFC and I think we should combine those
> and make one solution to satisfy all needs.
>
> Haris
>
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Leaf Li
> Sent: Tuesday, October 20, 2009 11:46
> To: mapguide-internals at lists.osgeo.org
> Subject: [mapguide-internals] The pinned connection
>
> It seems we confuse transaction and pinned connection. So before I answer
> you question, l would like to introduce sameness and difference between them
> a little bit.
>
> Sameness:
> 1. Both transaction and pinned connection will exclusively use one FDO
> connections in FDO connection pool.
> 2. Both transaction and pinned connection will maintain connection state
> information.
>
> Difference:
> 1. Transaction will guarantee that operations executed in one transaction
> are atomic. So it has commit and rollback method. But it doesn't make sense
> for pinned connection. Pinned connection just guarantee that operations use
> the same FDO connection. Pinned connection doesn't want commit and rollback
> operations.
> 2. One transaction may do something on the DRBMS server when opening a
> transaction. Changes in one transaction isn't visible for operations which
> aren't in the same transaction before committing transaction. It isn't true
> for pinned connection. Changes in one pinned connection can be visible for
> other users. The connection state information that transaction and pinned
> connection maintain is different.
>
> What we need is a pinned connection. Users can use the same connection when
> calling methods
> ExecuteSqlQuery
> ExecuteSqlNonQuery
> UpdateFeatures
> SelectFeatures
> SelectAggregate
> ......
>
> For example, we can call method ExecuteSqlNonQuery against Oracle feature
> source to execute the following statements,
> LOCK TABLE ...
> ALTER SESSION ...
> CONNECT AS SYSDBA
> ......
>
> Those connection state information should be bound to a users session. For
> example, after one user session execute "CONNECT AS SYSDBA", this FDO
> connection should be bound to this session and can't be used by other users
> session. Otherwise, other users may have SYSDBA access right by using this
> FDO connection.
>
> This is the motivation of this RFC. It happen we use a nested transaction
> (MapGuide and FDO doesn't support nested transaction) sample and a nested
> data reader sample to show intention. So it may result in some confusing
> between transaction and pinned connection. Hope it is clear now.
>
> Thanks,
> Leaf Li
>
>
>   
>> Hello,
>>
>> My general advice is to avoid extending the API if there is any possible
>> logic which
>> do the job without exposing such internal states to the API.
>> So the question is if there is any business logic to decide this?
>>
>> I think one important thing missing in RFC86 is a complete list of use
>> cases this is wanted for.
>> (Maybe we should focus more on the use cases to be part of any RFC to
>> help further evaluation of it. )
>>
>> The use case/example in RFC86 looks to me like a transaction which
>> should reuse the same FDO connection
>> for the same user session. That was the start of my comment. I believe
>> using a dedicated FDO connection makes
>> a lot of sense for the Transactions.
>>
>> So what are the other use cases?
>>
>> We can easily agree that the FDO connection pool needs to be a little
>> smarter than it is.
>> Exposing those things to the API feels wrong.
>> So can someone help to complete the use cases so we can write the logic
>> to manage the FDO connection pool?
>>
>>
>> In this case we have to consider single user and multi user scenarios.
>>
>> Haris Kurtagic wrote:
>> Leaf, UV,
>>
>> To be able to use only one "pinned" connection in session is unnecessary
>> restriction to application.
>>
>> I think UV comment was correct that this RFC is connected to RFC78 (thanks
>> UV to pointing it out).
>> Instead of adding parameter to set transaction in execute sql functions it
>> should be better to use connection as parameter.
>> On that connection transaction is started or not, which depends of'course
>>     
> on
>   
>> application.
>>
>> In such case you can any number of transaction, readers, "pinned"
>> connections whatever.
>> Application is in control how and for what wants to use them. In that case
>> there is no confusion what is pinned what is in transaction etc...
>>
>> Haris
>>