[Gdal-dev] Content length field mismatch in shapefiles

[Roger Bivand]

On Mon, 1 May 2006, Frank Warmerdam wrote:

> Roger Bivand wrote:
> > Shall I write a very small shapelib contrib function/program to test for 
> > this malformity?
> 
> Roger,
> 
> If you did so, I would add it.  Alternatively SHPOpen() could recover from
> missing .shx files by rebuilding one.

Frank,

Here are the two functions (I'm not sure where they might best live):


int SHPCheckSHX( SHPHandle hSHP )
/* checks file length against implied file length in *.shx to guard */
/* against overrun */

{
    int result = 0;
    if ((hSHP->panRecOffset[hSHP->nRecords-1] + 
	hSHP->panRecSize[hSHP->nRecords-1] + 8) > hSHP->nFileSize)
	    result = 1;
    return( result );
}

int SHPCheckSHX_Geolytics( SHPHandle hSHP )
/* checks for Geolytics Inc. off by 8 bytes malformity in *.shx */

{
    int i, result;
    for (i=1, result=0; i < hSHP->nRecords; i++)
	if (hSHP->panRecOffset[i] != (hSHP->panRecOffset[i-1] + 
	    hSHP->panRecSize[i-1])) result++;

    return( result );

}

and rather than a separate utility, maybe this:

    if ( SHPCheckSHX( hSHP ) != 0 ) {
	printf( "Mismatch between shp file length and shx file.\n" );
	if ( SHPCheckSHX_Geolytics( hSHP ) == 0 )
	    printf( "Off-by-8 malformity in shx file.\n" );
    }

after about line 74 in contrib/shpinfo.c; I've tested the functions but 
not the suggested addition to shpinfo.

Your idea of having SHPOpen() reconstruct a missing *.shx is of course the 
best choice, because no argument passing is needed, just knowledge in the 
community that if shpinfo reports a length mismatch, they should 
rename/delete the shx file and try again, which would work for OGR nicely.

Doing this would be a substantial intervention in SHPOpen(), though, 
setting a no_SHX flag, and conditioning on it from there on. Do we need to 
write an shx file out, is it sufficient simply to populate the SHPHandle 
correctly?

Best wishes,

Roger

> 
> Best regards,
> 

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no