[gdal-dev] shapefile enhancements

[Jukka Rahkonen]

Jan Heckman <jan.heckman <at> gmail.com> writes:

> 
> 
> Hi,It appears I have to do some homework on ogr's shapefile functions as
it stands now.
> 
> 8GB: If interoperability is more of a priority than capacity, that's a
valid consideration. I've not really needed anything > 4GB so far.
> 
> Delete:
> By delete I mean leaving the information in the file but (shapefile)
taking it out of the index chain (.shx), and .dbf, marking the record with
an asterisk in its firs byte.
> 
> As far as arcgis, I did a delete in this way and tried to load it. When I
do not reduce the record count in the dbf header, arcgis will not load it;
when I do reduce the record count in the header, arcgis will load the
shapefile but the attributes will not match the shapes. As a cross-check,
you can open the .dbf in open office or excel: the delete will be recognized.
>  
> My guess is that arcgis maps the shaperecords to the physical records of
the dbf only.
> 
> To allow use of the shapefile in arcgis,  I have to compact the .dbf. The
shape will then be handled correctly.
> 
> A recipe to try this out:
> create a new empty point shapefile, load it in arcgis. Using arccatalog to
create the shapefile, it will have a single ID integer attribute. That's the
starting point.
> Create 3 points and give them ID's 1 - 3.
> Now to 'delete' the second record using a diskeditor:
> Copy the shapefile. Open the .shx. The .shx has a header and records
consisting of offset-length pairs. A pair takes 8 bytes. Change the 2nd
offset to be identical to the last (00000040 -> 0000004E). Diminish the
filelength indicator in the header (offset 0x18) by 4 (0000003E to
0000003A). Copy the file, except the last 8 bytes to the new .shx file.
> DBF: open in editor, change the first byte of the second record (at offset
0x48) to an asterisk. The recordcount in the header is at offset 4 (little
endian).
> 
> Load in arcgis, will fail.
> Change  the dbf-recordcount to 2, load and open the attribute table. The
file will load. The ID's shown will be 1 and 2. The latter should have been 3.
> Conclusion: shape deletes are possible, .dbf not. Compacting the .dbf
will, of course, repair things.


Hi,

Fast delete feels like a nice trick which may have internal use during an
editing session. However, I consider that delivering such shapefiles for
customers would be dangerous. For example, there are programs like OpenJUMP
which do not seem to care about the .shx part at all when opening a shapefile.

-Jukka Rahkonen-

> Best regards,
> Jan