[QGIS-Developer] QGIS/OGR: FeatureIds reassigned on write to data provider?

[Even Rouault]

On mardi 11 juillet 2017 14:00:28 CEST Sandro Mani wrote:
> On 10.07.2017 13:58, Even Rouault wrote:
> > > I did some playing around and came up with some FeatureId mapping code
> > > 
> > > which is pretty space efficient (basically requiring just
> > > 
> > > 3*nDeletedFeatures entries), see below.
> > > 
> > > 
> > > 
> > > The code however assumes that the gaps in the fid-sequence are filled by
> > > 
> > > decreasing the fids after the gap, which may be pretty specific to the
> > > 
> > > OGR/Shapefile behaviour.
> > 
> > This should clearly be restricted to Shapefiles. Other drivers don't
> > have this compaction logic and will let happily holes in the feature ids.
> > 
> > > Hence in my view such code should really belong
> > > 
> > > into the OGR provider (or perhaps even ogr itself) and not in the
> > > 
> > > plugin. Thoughts?
> > 
> > Regarding putting that in OGR itself, there's no such API for that
> > right now, and I'm not completely clear of the value to add one
> > (specific use case, for a single driver)
> > 
> > Perhaps the QGIS OGR provider would be a better place for now.
> 
> So that kinda works, but repacking still causes dreadful performance
> (say after fixing a couple of hundred "minimal area" errors, which
> results in the small features getting deleted, the shapefile is repacked
> after every delete).
> 
> So I'd really need a way to freeze repacking, even just for the timespan
> the plugin is busy fixing the errors. Unless there are some better
> ideas, I'll look at adding corresponding methods. A cool name would be
> beginTransaction/endTransaction which is kinda generic and could make
> sense for other providers as well, but unfortunately it collides with
> the existing transaction logic (QgsTransaction) which is about
> multi-layer transaction groups...

As you venture into the OGR provider, you'll see related things :

- there's an internal to the OGR provider startTransaction() / commitTransaction()  that is 
used only for SQL based drivers, and in the scope of a single operation like addFeatures() / 
deleteFeatures() /..

- there's a enterUpdateMode() / leaveUpdateMode() pair respectively called by 
QgsVectorLayer::startEditing() / QgsVectorLayer::commitChanges(). The purpose of that is to 
respectively re-open a read-only file as read-write, and re-open it from read-write back to 
read-only. This is for scenarios involving QGIS with other GIS open at the same time on th 
same file: if QGIS opens it in read-write mode, that prevents the other software from 
opening it (was initially for MapInfo).
We could potentially imagine to defer the repacking at leaveUpdateMode() time, actually in 
the close() method which is called when the reference counter of the update mode drops to 
zero (*), instead of doing it at the end of each operation.

Even

(*)  in the case changes are done directly on the provider objet, without using 
QgsVectorLayer::startEditing() / QgsVectorLayer::commitChanges(),  recompaction should be 
done at the minimum at close() time (with GDAL 2.2 or later, this wouldn't even be 
necessaryas automatically done by the OGR Shapefile driver, but that would matter for older 
versions).

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20170711/ecdde54a/attachment-0001.html>