[gdal-dev] Precission problem with atof

[Mateusz Loskot]

Jorge ArĂŠvalo wrote:
> 2009/7/23 Mateusz Loskot <mateusz at loskot.net>:
>> Jorge ArÄ&#65533;Ĺ valo wrote::
>>> Hello,
>>>
>>> Context: GDAL WKT Raster Driver
>>>
>>> While creating the Dataset, I execute this query:
>>>
>>> SELECT srid, ST_AsText(extent), pixelsize_x, pixelsize_y, blocksize_x,
>>> blocksize_y FROM raster_columns WHERE r_table_name = 'table' AND
>>> r_column = 'rast'
>>>
>>> I have loaded a 3-band tiff file with gdal2tiles. Its pixel size is
>>> 0.9. If I execute this line (over the result of the previous query):
>>>
>>> Â fPixelSizeX = atof(PQgetvalue(hPGresult, 0, 2));
>>>
>>> I get 0.899999976158142 instead of 0.9.
>>
>> Good.
>>
>>> The string obtained wiht PQgetvalue is "0.9", but, for any reason,
>>> atof transform this string in 0.899999976158142.
>>
>> http://docs.sun.com/source/806-3568/ncg_goldberg.html
>>
>>> So, seems to be a
>>> precission problem. think the string used by atof has any problem, but
>>> I'm not sure
>>
>> Most computers suffer of this problem.
>>
>>> What should I do?
>>
>> You need to learn how to live with that problem :-)
>>
>>> I tried to change atof(nptr) for strtod(nptr,
>>> (char**)NULL), but it didn't work...
>>
>> And it won't help.
>>
>> Best regards,
>> --
>> Mateusz Loskot
>> http://mateusz.loskot.net
>>
>>
>
> Yes, a stupid problem. I used float instead of double.

It's not a problem of float vs double.
Replacing float with double does not guarantee that
value 0.9 will be represented in memory as 0.9.
It still can be 0.89999997...

The float-point term in the docs I suggested does not mean C/C++ type float.
It means float-point number in arithmetic, which in C/C++ can be float or
double.

Best regards
-- 
Mateusz Loskot
http://mateusz.loskot.net