[gdal-dev] Numeric accuracy

[Ari Jolma]

Ari Jolma kirjoitti 11.11.2017 klo 12:05:
> I have been bitten twice, once with ArcGIS and now with Rasdaman WCS, 
> with numeric accuracy.
>
> I'm making a data request on the corner of the bounding box, let's say 
> it has minimum X of 75042.7273594. I'm setting my minX to that value 
> and I'm enforcing it to that value with MAX (this was introduced 
> because of a case with ArcGIS). The I print that to the request with 
> "%.18g" (it was "%.15g" earlier but I changes it to that because of 
> ArcGIS) and the result is 75042.7273593999998, which is not good for 
> Rasdaman, since it is formally less than 75042.7273594. Although, in gdb
>
> (gdb) p 75042.7273594 > 75042.727359399999
> $7 = false
>
> Any ideas how to detect/prevent these kinds of situations? Keep the 
> checks and go back to "%.15g"? Not ok with ArcGIS.

I made a small function to convert the floating point numbers to strings 
(using "%.18g") and then comparing the strings. Seems to work but it 
requires storing the coverage envelope in the DescribeCoverage XML as 
strings and using those. That's since simply reading "75042.7273594" 
into a double and then printing it out with "%.18g" changes it into 
"75042.727359399999".

Ari

>
> Ari
>
>