[gdal-dev] Precission problem with atof
Jorge Arévalo
jorge.arevalo at gmail.com
Wed Jul 22 19:30:29 EDT 2009
Hello,
2009/7/23 Adam Nowacki <nowak at xpam.de>:
> Jorge Arévalo wrote:
>>
>> fPixelSizeX = atof(PQgetvalue(hPGresult, 0, 2));
>>
>> I get 0.899999976158142 instead of 0.9.
>
> 0.899999976158142... is the value closest to 0.9 that a single precision
> float variable can store.
> Using 'double' should give 0.90000000000000002220...
>
Wow, How could I be so stupid... I defined pixel size as float... I'm
gonna correct it and test again.
Many thanks!
Best regards
Jorge
More information about the gdal-dev
mailing list