[Qgis-user] New Shapefile Features Fail to Save
Nicolas Cadieux
nicolas.cadieux at archeotec.ca
Wed Feb 3 07:01:04 PST 2016
Hi Uwe,
This is an email written by Even Rouault from the GDAL dev team. It partially explains the difficulties with deleting geometries in shape files. (Read the last bit about QGIS also). I thought you would like to read it.
Nicolas
.....
Hi,
There have been some recent discussion on the qgis list about an old ticket
https://hub.qgis.org/issues/11007
Basically the issue seems to be that a lot / most non-shapelib / non-OGR based
shapefile readers don't understand the way OGR delete features in shapefiles.
When OGR/shapelib deletes a feature, it simply marks the corresponding record
in the DBF as deleted (technically putting a '*' character in the first byte of
the DBF record) and that's all. Very fast and OGR handles that consistently
(with the small restriction that the feature count reports the deleted
features as still existing, but iteration or getting features by id do not
report them)
This way of deleting a DBF record is the documented one :
http://www.clicketyclick.dk/databases/xbase/format/dbf.html#DBF_STRUCT
"""
Deleted flag:
Value Description
2Ah (*) Record is deleted
20h (blank) Record is valid
"""
However other GIS packages, and among others, a famous proprietary one - let's
call it "LineGIS" - when reading such shapefiles do not recognize the deleted
feature as deleted and display both the geometry and attributes. More
annoying, when "LineGIS" deletes another record in such a shapefile and saves
the result, the shapefile can no longer be opened afterwards with an error
message reporting an inconsistency in number of shapes w.r.t number of records
(and on inspection, the shp/shx indeed contain N - 1 records and the dbf N -
2, so it looks like it would be semi-aware of deleted DBF records)
When "LineGIS" starts with a "clean" shapefile and deletes a record in it, it
removes the corresponding entries in the .dbf, .shp and .shx files, which is
the result of the REPACK operation the shapefile driver can do if explicitly
asked.
"LineGIS" isn't the only one to have troubles with deleted DBF records. From
what I can see GeoTools (just picking a random example) only fully handle them
since 2014 :
https://osgeo-org.atlassian.net/browse/GEOT-4539
https://github.com/geotools/geotools/commit/e7333ccb284d137f3240ce5d0d09b3d7195f1890
The shapefile specification (
http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf ) doesn't mention
about how deleted records should be handled. Particularly if the requirement
"The table must contain one record per shape feature" (page 25) allows DBF
records marked as deleted... Anyway the theory/spec and the practice are 2
different things.
What surprises me is such an issue didn't raise more loud complaints before as
the OGR / shapelib behaviour has been the same since forever AFAIK.
I'm wondering if OGR shouldn't automatically run REPACK when closing a
shapefile when deletions (as well as edit operations of existing features
leading to holes in the .shp) have happened. The side effect of this would be a
slower closing (creation only scenarios wouldn't be affected) and a renumbering
of the FID of features after the deleted feature(s).
Thoughts ?
(Regarding the QGIS issue, as QGIS explicitly runs REPACK after
edition/deleting, it is not clear why the issue would persist. But some
reports might be with older QGIS/GDAL versions)
Even
Nicolas Cadieux M.Sc.
Les Entreprises Archéotec inc.
8548, rue Saint-Denis Montréal H2P 2H2
Téléphone: 514.381.5112 Fax: 514.381.4995
www.archeotec.ca
On Feb 3, 2016 3:29 AM, "Uwe Fischer [via OSGeo.org]" <ml-node+s1560n5248609h40 at n6.nabble.com> wrote:
>
> Spring,
>
>
>
> it seems to me like there are various errors in editing shapefiles in QGIS 2.10 and 2.12. Some weeks ago, I reported an error that does not allow for proper feature deleting. The shapes I deleted were gone in QGIS, but they were still present in ArcMap.
>
>
>
> Matthias Kuhn wrote in this context:
>
>
>
> Hi
>
> I just received a confirmation that Marco is working on a fix for this issue and targets the 2.14 release.
>
> GDAL 2.0 would still be nice to also have the geometry issue fixed :)
>
>
>
> Best regards
>
> --
>
> Matthias Kuhn
>
>
>
>
>
> If it turns out that your problems come from the same origin, maybe it’s time now to disable shapefile editing in QGIS until those problems are solved.
>
>
>
> Maybe someone more competent than me can give information to the list on the troubleshooting status? QGIS is such an outstanding piece of software, we’d love to see that errors disappear!
>
>
>
>
>
>
>
> Mit freundlichen Grüßen,
>
>
>
> Uwe Fischer
>
>
>
> Von: Qgis-user [mailto:[hidden email]] Im Auftrag von Springfield Harrison
> Gesendet: Mittwoch, 3. Februar 2016 07:30
> An: Nicolas Cadieux <[hidden email]>
> Cc: qgis-user <[hidden email]>
> Betreff: Re: [Qgis-user] New Shapefile Features Fail to Save
>
>
>
> Thanks for those ideas. I can check off most of those items except upgrading. This is new development so likely computer based? Rather than QGIS.
>
> I'll look at the bug reports; this is a pretty serious error. Thank you . . . . . .
>
> Cheers . . . . . Spring
> Samsung Tab 4
>
> On Feb 2, 2016 7:15 PM, "Nicolas Cadieux" <[hidden email]> wrote:
>>
>> Hi,
>> I have done this in the past without errors in QGIS. I would upgrade to the latest version. Then to figure out where the problem is, I would:
>>
>> Make sure that both layer are polygon layers (and not lines and polygons), that both have the same CRS (along with the project CRS), that both have the same fields and that the "receiving layer" is in editing mode, and saved subsequently. I would limited the amount of open files and open plugins.
>>
>> Try copying polygons from a newly created file to an empty newly created layer in QGIS and not a shapefile that was previously manipulated in other softwares. All this could help you find the source of the problem.
>>
>> I would also look at the bug reports to see if it is an issue for that version.
>>
>> Hope this helps.
>>
>> Nicolas Cadieux M.Sc.
>> Les Entreprises Archéotec inc.
>> 8548, rue Saint-Denis Montréal H2P 2H2
>> Téléphone: <a href="tel:514.381.5112" target="_blank">514.381.5112 Fax: <a href="tel:514.381.4995" target="_blank">514.381.4995
>> www.archeotec.ca
>>
>> On Feb 2, 2016 8:46 PM, "Springfield Harrison [via OSGeo.org] " <[hidden email]> wrote:
>>>
>>> Hello Group,
>>>
>>> Am using QGIS 2.10.1 and have done so without significant problems for many months. I am attempting to create simple polygons in a shapefile by direct vertex entry or copying polygons from another shapefile layer and pasting them into the active layer.
>>>
>>> In almost every case, these features disappear. There is an entity entry in the attribute table but there are no polygons created. In one case a polygon was created but it was not the shape that I digitized; no more polygons could be created although the process for doing so appears to be working. No error messages were created. The CRS for the project and the shapefile are the same.
>>>
>>> The shapefile can be opened in another GIS but has no content.
>>>
>>> Here are some product details: QGIS version 2.10.1-Pisa QGIS code revision d20c5b7
>>>
>>> Thanks very much for any help you can render, Cheers . . . . . . . . Springfield Harrison
>>>
>>> _______________________________________________
>>> Qgis-user mailing list
>>> <a href="/user/SendEmail.jtp?type=node&node=5248589&i=0">[hidden email]
>>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-user
>>> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-user
>>>
>>> ________________________________
>>>
>>> If you reply to this email, your message will be added to the discussion below:
>>>
>>> http://osgeo-org.1560.x6.nabble.com/FW-QGIS-crashes-when-using-mouse-wheel-to-zoom-in-and-out-of-WMTS-services-tp5248434p5248589.html
>>>
>>> To start a new topic under Quantum GIS - User, email [hidden email]
>>> To unsubscribe from Quantum GIS - User, <a href="http://osgeo-org.1560.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4125267&code=bmljb2xhcy5jYWRpZXV4QGFyY2hlb3RlYy5jYXw0MTI1MjY3fDYzNDQ4MjQxNg==">click here.
>>> <a href="http://osgeo-org.1560.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml" style="font:9px serif">NAML
>>
>>
>>
>> ________________________________
>>
>> View this message in context: Re: New Shapefile Features Fail to Save
>> Sent from the Quantum GIS - User mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> Qgis-user mailing list
>> [hidden email]
>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-user
>
>
> _______________________________________________
> Qgis-user mailing list
> [hidden email]
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-user
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://osgeo-org.1560.x6.nabble.com/FW-QGIS-crashes-when-using-mouse-wheel-to-zoom-in-and-out-of-WMTS-services-tp5248434p5248609.html
> To start a new topic under Quantum GIS - User, email ml-node+s1560n4125267h38 at n6.nabble.com
> To unsubscribe from Quantum GIS - User, click here.
> NAML
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/FW-QGIS-crashes-when-using-mouse-wheel-to-zoom-in-and-out-of-WMTS-services-tp5248434p5248691.html
Sent from the Quantum GIS - User mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20160203/056b9ee3/attachment.html>
More information about the Qgis-user
mailing list