[gdal-dev] Precission problem with atof

[Jorge Arévalo]

2009/7/23 Mateusz Loskot <mateusz at loskot.net>:
> Jorge ArĂŠvalo wrote:
>> 2009/7/23 Mateusz Loskot <mateusz at loskot.net>:
>>> Jorge ArÄ&#65533;Ĺ valo wrote::
>>>> Hello,
>>>>
>>>> Context: GDAL WKT Raster Driver
>>>>
>>>> While creating the Dataset, I execute this query:
>>>>
>>>> SELECT srid, ST_AsText(extent), pixelsize_x, pixelsize_y, blocksize_x,
>>>> blocksize_y FROM raster_columns WHERE r_table_name = 'table' AND
>>>> r_column = 'rast'
>>>>
>>>> I have loaded a 3-band tiff file with gdal2tiles. Its pixel size is
>>>> 0.9. If I execute this line (over the result of the previous query):
>>>>
>>>> Â fPixelSizeX = atof(PQgetvalue(hPGresult, 0, 2));
>>>>
>>>> I get 0.899999976158142 instead of 0.9.
>>>
>>> Good.
>>>
>>>> The string obtained wiht PQgetvalue is "0.9", but, for any reason,
>>>> atof transform this string in 0.899999976158142.
>>>
>>> http://docs.sun.com/source/806-3568/ncg_goldberg.html
>>>
>>>> So, seems to be a
>>>> precission problem. think the string used by atof has any problem, but
>>>> I'm not sure
>>>
>>> Most computers suffer of this problem.
>>>
>>>> What should I do?
>>>
>>> You need to learn how to live with that problem :-)
>>>
>>>> I tried to change atof(nptr) for strtod(nptr,
>>>> (char**)NULL), but it didn't work...
>>>
>>> And it won't help.
>>>
>>> Best regards,
>>> --
>>> Mateusz Loskot
>>> http://mateusz.loskot.net
>>>
>>>
>>
>> Yes, a stupid problem. I used float instead of double.
>
> It's not a problem of float vs double.
> Replacing float with double does not guarantee that
> value 0.9 will be represented in memory as 0.9.
> It still can be 0.89999997...
>
> The float-point term in the docs I suggested does not mean C/C++ type float.
> It means float-point number in arithmetic, which in C/C++ can be float or
> double.

Ok. I changed float for double and now fPixelSize has 0.9, as I want.
But I understand, as you said, that "replacing float with double does
not guarantee that value 0.9 will be represented in memory as 0.9".
So, I'll read again the docs. I find them really interesting.

Many thanks again, Mateusz!

Best regards
Jorge



>
> Best regards
> --
> Mateusz Loskot
> http://mateusz.loskot.net
>
>