[gdal-dev] Precission problem with atof

Jorge Arévalo jorge.arevalo at gmail.com
Wed Jul 22 19:30:29 EDT 2009


Hello,

2009/7/23 Adam Nowacki <nowak at xpam.de>:
> Jorge Arévalo wrote:
>>
>>  fPixelSizeX = atof(PQgetvalue(hPGresult, 0, 2));
>>
>> I get 0.899999976158142 instead of 0.9.
>
> 0.899999976158142... is the value closest to 0.9 that a single precision
> float variable can store.
> Using 'double' should give 0.90000000000000002220...
>

Wow, How could I be so stupid... I defined pixel size as float... I'm
gonna correct it and test again.

Many thanks!

Best regards
Jorge


More information about the gdal-dev mailing list