[Gdal-dev] new RFC 13: Improved Feature Insertion/Update/Delete Performance in Batch Mode

[Frank Warmerdam]

Baumann, Konstantin wrote:
> Hi!
> 
> I would like to announce the new RFC 13:
> <http://trac.osgeo.org/gdal/wiki/rfc13_createfeatures>
> 
> I would like to add some new methods to OGRLayer for "batched"
> inserting/updating/deleting features. These virtual methods allow OGR
> drivers to implement optimized handling of these batched operations
> (e.g. resulting in a performance boost).
> 
> For example, I was able to improve the "batched insertion" scenario for
> the MySQL driver from ~40 features per second to something about
> 800-1.500 features per second; for layers with >=50.000 features (with
> and wihtout geometry)...
> 
> Any further comments and suggestions are wellcome.

Kosta,

I presume the optimization in the mysql driver was accomplished by doing
all the features in a single CreateFeatures() call as a single transaction.
Is that right?

There are other mechanisms to accomplish this - based on the OGR transaction
semantics.  For instance, I think the postgres and oracle drivers implement
ogr transaction semantics and can do a bunch of CreateFeature() calls in
a single transaction.  The ogr2ogr program already defaults to doing feature
creation in groups of 200 by default, with transaction semantics around this.

On the other hand, the transaction semantics are quite complicated to
implement properly in drivers, which is why I suspect they haven't been
done in the MySQL driver.  So if we were to proceed with this RFC I think
it would be because it provides a simplier approach to grouping create
features than the transaction semantics.

But because this introduces an extra interface and the related complexity
I'm hesitant to support it when there is another way to accomplish the
same thing.

One other small concern.  I think it can be inconvenient to implement
swig wrappers for interfaces taking arrays of objects.  Can any of the
swig folks comment on this?

> What is the procedure to start voting on that?

Normal practice would be to allow some discussion and revision and then to
call a formal motion to adopt the RFC, which would start the 2 business day
voting period.  You would need +1's from at least two GDAL PSC members and
no objections (-1's) to pass the RFC.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org