[gdal-dev] Invalid node descriptor size in .sbn

[Even Rouault]

Le mercredi 12 février 2014 15:27:05, Dennis Gocke a écrit :
> Hi Even,
> 
> Thanks for the quick reply.
> 
> regarding the last part:
> Here is some pseudo code to illustrate:
> I'm normally using the C# wrapper so Failures result in an exception.
> 
> When doing the following:
> OpenShape()
> GetSpatRef()
> SetSpatialFilter()
> GetNextFeature() -> Exception "Invalid node descriptor size in .sbn"
> 
> when doing the same just without GetSpatRef():
> OpenShape()
> SetSpatialFilter()
> GetNextFeature() -> No Exception
> 
> This seemed strange to me.

Yes, for me too without deeper investigation. Well you shouldn't worry anymore 
with the fix.

> 
> With the increased Maximum value it is probably unlikely that this will
> still happen,

Yes that should be OK for shapefiles up to 2^24 * 8 = 134 millions shapes. 
Although I'm wondering if that area of the code shouldn't be modified in a 
different way, but the new value should be OK for now.

> but should this Problem with the SBN really result in a
> Failure or wouldn't be a Warning enough?

Well that's a self consistency test. So when it doesn't pass, a failure is 
appropriate. Nominal working is sufficiently hard to implement, so optimizing 
behaviour in non-nominal cases ...

> 
> Kind regards,
> Dennis
> 
> 
> 
> On Wed, Feb 12, 2014 at 2:53 PM, Even Rouault
> 
> <even.rouault at mines-paris.org>wrote:
> > Le mercredi 12 février 2014 13:40:38, Dennis Gocke a écrit :
> > > Hi,
> > > 
> > > 
> > > 
> > > I found a problem with the shape file driver that occurs when reading
> > > features with a spatial filter from a large shapefile that has a
> > > spatial index (sbn).
> > > 
> > > 
> > > 
> > > The problem can be reproduced with this shapefile
> > > http://geofuse.geoeye.com/static/shp/GE1_2012.zip but it's not only
> > > this shapefile, I have others that show the problem, all of them are
> > > rather
> > 
> > big.
> > 
> > > Recalculating the spatial index (with ArcGIS) doesn't solve the
> > > problem, the files seem to be valid.
> > > 
> > > 
> > > 
> > > When calling CPLGetLastError..() after the first GetNextFeature() a
> > 
> > failure
> > 
> > > is returned with the message "Invalid node descriptor size in .sbn :
> > > 503800".
> > > 
> > > It seems one could just ignore the Failure and continue reading the
> > > features. Everything seems fine (although it probably did not use the
> > > spatial index for filtering).
> > > 
> > > Perhaps a Warning instead of a Failure would suffice.
> > > 
> > > 
> > > 
> > > The error is generated in sbnsearch.c because "nNodeDescCount>
> > 
> > nMaxNodes".
> > 
> > > It seems the maximum tree depth of 15 is not enough.
> > > 
> > > Perhaps the maximum should be increased.
> > 
> > Yes. Fixed by http://trac.osgeo.org/gdal/ticket/5383. The limitation to
> > 15 was
> > not really justified.
> > 
> > > While testing this I also noticed a strange behavior. Sometimes
> > > GetLastError..() returned a failure and sometimes it did not. When not
> > > calling layer.GetSpatialRef() before calling GetNextFeature() the first
> > > time no failure was returned although it still of course had problems
> > > reading the sbn and "hSBN->sHooks.Error( szErrorMsg );" was still
> > > called.
> > 
> > Hum not sure to follow you completely. But sometimes the error flag may
> > be cleared. The first time you call GetSpatialRef() it reads the .prj
> > file and I
> > can see that for some reason the routine that does it clears the error
> > flag.
> > 
> > > This happens with the current development Version, but is also there in
> > 
> > the
> > 
> > > 1.10.0 Version.
> > > 
> > > Best Regards,
> > > 
> > > Dennis
> > 
> > --
> > Geospatial professional services
> > http://even.rouault.free.fr/services.html

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html