[Proj] Proj4 Bug (rtodms)

[Gerald I. Evenden]

On Thursday 02 November 2006 7:34 am, cp wrote:
> due to floating point precision, there is a bug in source code of rtodms.c
>
> the portion of code
>
>   r = floor(r * CONV + .5);
>   sec = fmod(r / RES, 60.);
>   r = floor(r / RES60);
>   min = fmod(r, 60.);
>   deg = r / 60.;
>
> must be replaced by
>
>   r = floor(r * CONV + .5);
>   sec = fmod(r / RES, 60.);
>   r = floor(r / RES60);
>   min = fmod(r, 60.);
>   r = floor(r / 60.);
>   deg = r;
>
> the conversion of 0.06981317007977319578 (4 degree in radian)  is 3d with
> the first code (wrong) and  4d with the second (true).

So far, I have not found any way to reproduce your problem and consequently 
find no reason to change code.  However, I do have a couple of questions:

1. on what hardware (chip) was this problem created,

2. what compiler/library was employed and

3, level of optimization specified.

Interestingly, r before being divided by 60 does not have a fractional 
component (exactly equal to 240.).  The implication is that the floating 
point processor produces a "ragged" result where the mantissa is not 
rational.  That is, something like 3.9999... is produced.  The floor 
operation should still have produced 3 as the unchanged code did with simple 
conversion to integer with the memory assignment of the division operation.
I suspect something is gummed up with the handling of the extra bits that are 
part of the mantissa in the floating point hardware.

Lastly, the problem line has been handle by:
     deg = (int)(r / 60.);
for some time in the libproj4 library.  The cast was necessary to keep the 
compiler from complaining during -Wall runs.

Lastly, rtodms is deprecated and will be replaced with rad2dms which allows 
for greater format flexibility and is (unlike rtodms) thread proof.
-- 
The whole religious complexion of the modern world is due
to the absence from Jerusalem of a lunatic asylum.
-- Havelock Ellis (1859-1939)  British psychologist