[QGIS-trac] [Quantum GIS] #3395: Field Calculator Integer Conversion Errror

[Quantum GIS]

#3395: Field Calculator Integer Conversion Errror
------------------------------------------------+---------------------------
   Reporter:  alexckp                           |              Owner:  jef          
       Type:  bug                               |             Status:  new          
   Priority:  major: does not work as expected  |          Milestone:  Version 1.7.0
  Component:  Vectors                           |            Version:  Trunk        
   Keywords:                                    |   Platform_version:               
   Platform:  Windows                           |           Must_fix:  No           
Status_info:  0                                 |  
------------------------------------------------+---------------------------
 I tried to use FC to extract an integer number using the following
 formula:

 (SURV_BRG * 1000) - to int (to int(SURV_BRG * 100) * 10)

 If the number is 62.233, the above formula will correctly return the
 number 3. However, when I encounter certain numbers, e.g. the number 65.1,
 the result is 9.99999999999272 instead of 0. I think this has to do with
 the to int function not returning a true integer and would appreciate if
 this problem could be solved.

-- 
Ticket URL: <http://trac.osgeo.org/qgis/ticket/3395>
Quantum GIS <http://qgis.org>
Quantum GIS is an Open Source GIS viewer/editor supporting OGR, PostGIS, and GRASS formats