[Qgis-developer] Warning - shapefile corruption on 2.16/master and how to recover

[Even Rouault]

Le samedi 17 septembre 2016 00:00:55, Jeff McKenna a écrit :
> On 2016-09-16 3:53 PM, Even Rouault wrote:
> > One of my hypothesis is some bad interaction with the "connection pool".
> > Could you (or anyone) open the attribute table before doing the edits,and
> > do the edits and commit them no more than 60 seconds afterwards ? I'm
> > not sure but this might perhaps also require the shapefile to be big
> > enough (several thousands of features).
> 
> Here was my next test:
> 
> - download OSM land polygons for the world, unsplit:
> http://data.openstreetmapdata.com/land-polygons-complete-4326.zip
> 424MB   568,635 features  <--- big test!
> 
> - add shapefile to QGIS 2.16.2 view
> 
> - open attribute table
> 
> - start editing
> 
> - after a while waiting for nodes to appear as QGIS works, message is
> displayed of "qgis-bin.exe has stopped working"

I've tried your file 2.16.3 in a Win7 VM with 4GB and trying to reproduce your 
actions but didn't get a crash. I managed painfully to move a node of the 
coastline of western part of France. Note that I used a 64 bit build, and I 
noticed that the memory consumption goes up to 3 GB when you enable the Node 
tool (goes down to 300 MB once you finish all editing actions), so if you use a 
32 bit build, a crash is not surprising. 
All the actions are super slow, particularly making a node active and moving 
it. There are clearly issues, with algorithms, data structures and memory use, 
when dealing with huge polygons.
I also noticed that there's apparently always a core busy until you finish 
editing.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com