[gdal-dev] Lost in decimals... geotransform in python 2.7

[Even Rouault]

On dimanche 9 décembre 2018 11:42:27 CET Nicolas Cadieux wrote:
> Hi,
> 
> I am trying to get the four corners of a raster the size of Canada (very
> big therefore decimals count).  If I get the geotransform, I get:
> 
> geotransform: (-132.00041666666667, 0.0008333333333333334, 0.0,
> 53.000416666666666, 0.0, -0.0008333333333333334)
> In Python 2.7:
> print (geotransform[0]) ->               -132.000416667
> print float(geotransform[0]) ->       -132.000416667
> print str(geotransform[0]) ->           -132.000416667
> print Decimal(geotransform[0]) ->
> -132.000416666666666287710540927946567535400390625
> In Qgis3x (on Windows64), I get:    -132.0004166666666663 in the layer
> properties.
> Qgis looks like the end result of:
> getcontext().prec = 19
> print Decimal(geotransform[0])/Decimal(1)
> 
> Must I use "Decimals" to deal with the geotransform output? Using float
> will truncate the decimal place and I may end up with an extra pixel row
> or column because of the size of the raster. So what is the proper way
> to work with this?

Nicolas,

Using float is perfectly fine. Internally GDAL stores the geotransform 
components as IEEE-754 64bit numbers, and Python float is able to deal with 
that (it uses this representation internally on platforms you care about). It 
is just that the formatting functions (str, etc.) you use to convert the 
number to string truncate the number of decimals. If you use repr() instead 
you'll have more decimals

>>> repr(-132.00041666666667)
'-132.00041666666667'

Anyway, when dealing with floating point numbers, independently of their 
decimal representation, you must be aware that they don't necessarily capture 
perfectly the "real" number ( here -(132 + 0.5 / 1200) would be the exact 
number ), so you must be ready to add appropriate rounding in computations 
wherever needed.

Use string representation only as late as possible, or use 19 decimal 
significant figures ('%.19g' C formatting) to keep the original precision of a 
IEEE-754 64bit number.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com