[Proj] Re: Re: Re: Proj4 Bug (rtodms)

[Glynn Clements]

cp wrote:

> > As both the integer cast and floor round positive numbers downwards,
> > using double precision would result in a value slightly less than 4.0
> > being rounded down to 3, while extended precision would result in a
> > value slightly greater than 4.0 being rounded down to 4.
> 
> Not exactly !  for the C runtime library, even if the result is less than 
> 4.0, the floor gives 4.0 if the difference is less than the smaller number 
> in the precision, here if the difference is less than the double precision.
> (I have look at asm code of floor)

floor() takes a "double" argument, so an extended-precision (80-bit)
value would first be converted to a double by the FPU, which will
normally be done using round-to-nearest. floor() itself will then
round the value downwards.

This step might be skipped if the system's calling convention passes
FP arguments in registers, or if floor() is an intrinsic or inline
function. However, for a normal function call with arguments passed on
the stack (the actual stack, not the FPU register stack), the value
stored on the stack will only be 64 bits wide, so it has to be
converted first.

-- 
Glynn Clements <glynn at gclements.plus.com>