[gdal-dev] Numeric accuracy
Ari Jolma
ari.jolma at gmail.com
Sat Nov 11 02:05:50 PST 2017
I have been bitten twice, once with ArcGIS and now with Rasdaman WCS,
with numeric accuracy.
I'm making a data request on the corner of the bounding box, let's say
it has minimum X of 75042.7273594. I'm setting my minX to that value and
I'm enforcing it to that value with MAX (this was introduced because of
a case with ArcGIS). The I print that to the request with "%.18g" (it
was "%.15g" earlier but I changes it to that because of ArcGIS) and the
result is 75042.7273593999998, which is not good for Rasdaman, since it
is formally less than 75042.7273594. Although, in gdb
(gdb) p 75042.7273594 > 75042.727359399999
$7 = false
Any ideas how to detect/prevent these kinds of situations? Keep the
checks and go back to "%.15g"? Not ok with ArcGIS.
Ari
More information about the gdal-dev
mailing list