[mapguide-internals] The pinned connection

[UV]

The discussion is now getting the right direction.
It looks like we have a hierarchical relationship between dedicated 
connection and transaction context.

I like the example from Haris below. We need more like this!

We should consider all those use cases before the right decision can be 
made.
In particular the various multi user scenarios can be quite different.
I think it is rather important for such parts of the design to be very 
clear about the range of use cases involved.

E.g. on our server we are serving a huge map but read only.
I am sure this is a common but not the most complex user scenarios.

1. we have to make sure not to break the functionality in existing 
applications.
2. And we need to design this part of the system to be able to scale it up.

I am think the feature reader is one of the major bottleneck of this 
server so lets try to be very smart about this.
Scalability will become increasingly important in particular with cloud 
deployment in mind.


Badreddine Karoui wrote:
> In the web environment not having control over which connection to use is NOT a problem; It's a feature. The connection pooling/sharing is a scalability feature. However, that can be an issue/restriction for some special/edge and uncommon cases. For those cases, we have a special solution that can be used; temporary pin the connection. BTW. This is similar to pinning a .Net managed object when a unmanaged object need to keep a reference to the managed object.
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
> Sent: Tuesday, October 20, 2009 10:11 AM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> Difference is that using feature source id introduce problems we have now
> and which we are trying to solve. And problem is as stated in RFC that
> developer doesn't have control over which database connection is used when
> he wants to execute commands.
> If instead of feature source id, parameter would be connection then we
> wouldn't have this issues and RFC.
>
> Equivalent changes to API are already suggested in transaction RFC78. So
> changes shouldn't be so difficult.
> Just instead of transaction as suggested in that RFC we could use connection
> as parameter.
>
> I understand that making such changes will change existing API, I would much
> reader introduce new clear API then changing existing one to support one
> particular case and create existing API more difficult to understand and
> use.
>
>
> This is how I believe API should look like:
>
> Conn1 = Open(FeatureSourceId)
> Execute SQL(Conn1,..)
> Open(Conn2)
> Execute SQL(Conn2,..)
> Open(Conn3)
> Conn3.StartTransaction()
> Conn3.Committ()
> Execute SQL(Conn3,..)
> Close(conn3)
> Close(Conn2)
> Close(Conn1)
>
>
> Haris
>
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Tuesday, October 20, 2009 15:30
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] The pinned connection
>
> I'm not sure what is the difference between using the proposed fdo
> connection and using the existing feature source id. It seems that
> introducing the new connection will require more changes to the API since
> all the existing commands use feature source id. If the proposed connection
> will replace the feature source id, then that may not be acceptable since
> that means clients will be pinning all connections. If this is an addition
> to the API where both feature source id and connection will be used, then
> this is cumbersome. I think pinning a connection is an exception/edge case
> that should not be used unless it's absolutely necessary. As a result an
> explicit API as suggested by the RFC is appropriate in this case. 
>
> Badreddine
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris
> Kurtagic
> Sent: Tuesday, October 20, 2009 7:08 AM
> To: 'MapGuide Internals Mail List'
> Subject: RE: [mapguide-internals] The pinned connection
>
> I don't think we confused transaction with what is called pinned connection.
> I think unnecessary confusion and limitations are in application when pinned
> connection is hidden (assumed) to application as it is suggested in RFC.
>
> My suggestion is that developer can acquire FDO connection and then use that
> connection in ExecuteSql... functions. In that way developer have clear
> picture what is he doing with which FDO connection and can execute any
> combination of operations.
>
> Reason why I am mentioning RFC78 is because same solution ( but only for
> transactions ) was introduced in the RFC and I think we should combine those
> and make one solution to satisfy all needs.
>
> Haris
>
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Leaf Li
> Sent: Tuesday, October 20, 2009 11:46
> To: mapguide-internals at lists.osgeo.org
> Subject: [mapguide-internals] The pinned connection
>
> It seems we confuse transaction and pinned connection. So before I answer
> you question, l would like to introduce sameness and difference between them
> a little bit.
>
> Sameness:
> 1. Both transaction and pinned connection will exclusively use one FDO
> connections in FDO connection pool.
> 2. Both transaction and pinned connection will maintain connection state
> information. 
>
> Difference:
> 1. Transaction will guarantee that operations executed in one transaction
> are atomic. So it has commit and rollback method. But it doesn't make sense
> for pinned connection. Pinned connection just guarantee that operations use
> the same FDO connection. Pinned connection doesn't want commit and rollback
> operations.
> 2. One transaction may do something on the DRBMS server when opening a
> transaction. Changes in one transaction isn't visible for operations which
> aren't in the same transaction before committing transaction. It isn't true
> for pinned connection. Changes in one pinned connection can be visible for
> other users. The connection state information that transaction and pinned
> connection maintain is different.
>
> What we need is a pinned connection. Users can use the same connection when
> calling methods
> ExecuteSqlQuery
> ExecuteSqlNonQuery
> UpdateFeatures
> SelectFeatures
> SelectAggregate
> ......
>
> For example, we can call method ExecuteSqlNonQuery against Oracle feature
> source to execute the following statements, 
> LOCK TABLE ...
> ALTER SESSION ...
> CONNECT AS SYSDBA
> ......
>
> Those connection state information should be bound to a users session. For
> example, after one user session execute "CONNECT AS SYSDBA", this FDO
> connection should be bound to this session and can't be used by other users
> session. Otherwise, other users may have SYSDBA access right by using this
> FDO connection.
>
> This is the motivation of this RFC. It happen we use a nested transaction
> (MapGuide and FDO doesn't support nested transaction) sample and a nested
> data reader sample to show intention. So it may result in some confusing
> between transaction and pinned connection. Hope it is clear now.
>
> Thanks,
> Leaf Li
>
>
>   
>> Hello,
>>
>> My general advice is to avoid extending the API if there is any possible 
>> logic which
>> do the job without exposing such internal states to the API.
>> So the question is if there is any business logic to decide this?
>>
>> I think one important thing missing in RFC86 is a complete list of use 
>> cases this is wanted for.
>> (Maybe we should focus more on the use cases to be part of any RFC to 
>> help further evaluation of it. )
>>
>> The use case/example in RFC86 looks to me like a transaction which 
>> should reuse the same FDO connection
>> for the same user session. That was the start of my comment. I believe 
>> using a dedicated FDO connection makes
>> a lot of sense for the Transactions.
>>
>> So what are the other use cases?
>>
>> We can easily agree that the FDO connection pool needs to be a little 
>> smarter than it is.
>> Exposing those things to the API feels wrong.
>> So can someone help to complete the use cases so we can write the logic 
>> to manage the FDO connection pool?
>>
>>
>> In this case we have to consider single user and multi user scenarios.
>>
>> Haris Kurtagic wrote:
>> Leaf, UV,
>>
>> To be able to use only one "pinned" connection in session is unnecessary
>> restriction to application.
>>
>> I think UV comment was correct that this RFC is connected to RFC78 (thanks
>> UV to pointing it out).
>> Instead of adding parameter to set transaction in execute sql functions it
>> should be better to use connection as parameter.
>> On that connection transaction is started or not, which depends of'course
>>     
> on
>   
>> application.
>>
>> In such case you can any number of transaction, readers, "pinned"
>> connections whatever.
>> Application is in control how and for what wants to use them. In that case
>> there is no confusion what is pinned what is in transaction etc...
>>
>> Haris
>>     
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
>