[fdo-trac] #642: Sqlite filtering on number with more that 5 decimal fails

FDO trac_fdo at osgeo.org
Wed May 5 11:11:32 EDT 2010


#642: Sqlite filtering on number with more that 5 decimal fails
-----------------------------+----------------------------------------------
 Reporter:  bscott           |         Owner:  traianstanev
     Type:  defect           |        Status:  new         
 Priority:  major            |     Milestone:  3.5.0       
Component:  SQLite Provider  |       Version:  3.5.0       
 Severity:  3                |    Resolution:              
 Keywords:                   |   External_id:              
-----------------------------+----------------------------------------------
Comment (by traianstanev):

 I'm asking because floating point is different on the two, on 64 bit, all
 operations on double are always done to 64 bit, while on 32 bit, the
 floating point unit has 80 bit precision which doesn't always get
 truncated to 64 bits before comparisons are performed, depending on the
 compiler optimization settings used. So those operations can give
 different results on 32 vs 64.

-- 
Ticket URL: <http://trac.osgeo.org/fdo/ticket/642#comment:9>
FDO <http://fdo.osgeo.org/>
Feature Data Objects


More information about the fdo-trac mailing list