[fdo-internals] Re: FDO RFC 61 ready for review

[Traian Stanev]

Hi Orest,

The provider uses (old-style, non-nested) transactions internally to batch bulk inserts -- it hides that from the user, in order to be easier to work with the provider and to give maximum performance even for code which does not explicitly use transactions. This was initially necessary in order to plug into all the existing FDO code which was written with SDF in mind, and did not use explicit transactions.

However, if a transaction is already started by the user, the insert command will not start one (since the code was written when sqlite did not support nesting). This means that the user's transaction is honored by the provider. This works fine as long as one uses the FDO API, and not the SQL command. Then we also added a code path which used savepoints instead of transactions. It's still there, but turned off with an ifdef. However, with that code path, as far as I remember, there were invariably problems when the application gets in a state where a sqlite savepoint and a sqlite transaction get mixed up and blow away the savepoint stack or cause table locks -- I don't remember the details, but perhaps Romy does.

In order to support savepoints in FDO SQLite, it would probably be best to convert the code to only use the savepoint feature, at least when called from the FDO API. The current state of the logic is quite confusing, because it historically evolved before savepoints, then they were added later, then removed again -- so this RFC is an opportunity to clean that up, but it will require some more significant testing effort to make sure the change does not break client code by causing locked tables again.

All the relevant logic is concentrated in the StartTransaction/CommitTransaction/RollbackTransaction functions of the SltConnection class.

Traian


-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Orest Halustchak
Sent: Monday, May 23, 2011 8:32 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] Re: FDO RFC 61 ready for review

Hi Traian,

Can you explain a bit more how the provider uses transactions internally? In particular, if the user of the provider is controlling transactions themselves using begin/end transactions, how does the provider deal with that if there are inserts in between the begin/end transaction calls?

Thanks,
Orest.

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Monday, May 23, 2011 1:04 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] Re: FDO RFC 61 ready for review


We already had savepoints implemented in the SQLite provider, but removed them because they do not play nice with use of regular transactions. The provider uses transactions internally to (for example) batch inserts. So, whoever uses savepoints on top or in between of Fdo inserts with sqlite is asking for trouble.

Traian


-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Sam Wang
Sent: Sunday, May 22, 2011 10:40 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] Re: FDO RFC 61 ready for review

Hi Jackie,

In the proposed milestone(3.7.0), we will implement SQL server spatial and SQLite provider for this API enhancement.

Thanks,
Sam

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Jackie Ng
Sent: Monday, May 23, 2011 9:50 AM
To: fdo-internals at lists.osgeo.org
Subject: [fdo-internals] Re: FDO RFC 61 ready for review

Hi Sam,

What FDO providers will be the primary beneficiaries of this API enhancement?

- Jackie

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/FDO-RFC-61-ready-for-review-tp6392825p6392842.html
Sent from the FDO Internals mailing list archive at Nabble.com.
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals