[fdo-trac] #54: SHP: Loss of precision causing wrong spatial query results

[FDO]

#54: SHP: Loss of precision causing wrong spatial query results
----------------------------+-----------------------------------------------
   Reporter:  danstoica     |       Owner:  danstoica
       Type:  defect        |      Status:  new      
   Priority:  major         |   Milestone:  3.3.0    
  Component:  SHP Provider  |     Version:  3.2.0    
   Severity:  3             |    Keywords:           
External_id:                |  
----------------------------+-----------------------------------------------
 The nature of the problem is the fact the bounding box is stored both in
 SHP and IDX files. The SpatialExtents() aggregate function is using the
 first one which might not be exactly the same as the other one, causing
 primary filtering to miss features on the boundary.

 For very large numbers (say greater than 8 significant digits) or very
 high accuracy (like in lat/long datasets, where 6 digits accuracy is
 common) the tolerance used to compensate the precision loss is not enough
 (that is 0.001/2 or 0.0000001/2 for LL). That is, the difference between
 the extents stored in SHP (as doubles) and the extents stored in IDX (as
 floats) can be practically in any range. The solution is to compute this
 offset and adjust the bounding box accordingly before performing the
 primary filtering.

-- 
Ticket URL: <http://trac.osgeo.org/fdo/ticket/54>
FDO <http://fdo.osgeo.org/>
Feature Data Objects