[Proj] Using Proj.4 to compress a 10-parameter Molodensky-Badekas datum shift into 7 parameters

[support.mn at elisanet.fi]

Hello,

If I remember it right porj.4 only gives you no more than 12 cm accuracy all over
the world. That is written somewhere in the manuals. Additional to that:

proj.4 uses

#define PI		3.14159265358979323846

if that has anything to do with it. And of course the rounding at very small
angles might also play there and how the fpp handles the numbers.

VC++ tells us:

"Type double
Double precision values with double type have 8 bytes. The format is similar to the float format except that it has an 11-bit excess-1023 exponent and a 52-bit mantissa, plus the implied high-order 1 bit. This format gives a range of approximately 1.7E–308 to 1.7E+308 for type double.

Microsoft Specific —>

The double type contains 64 bits: 1 for sign, 11 for the exponent, and 52 for the mantissa. Its range is +/–1.7E308 with at least 15 digits of precision. 

END Microsoft Specific
"

so to be sure one should write PI with at least 17 decimals or more but 21 digits must be enough:

21 digits: 3.14159265358979323846
16 digits: 3.141592653589793

That can affect some routines and of course the accuracy of the computer used and how the compiler
handles the constants etc.

Regards: Janne

-----------------------------------------------------


Mikael Rittri [Mikael.Rittri at carmenta.com] kirjoitti: 

> In my first, wrongly computed example, I had a mismatch of about 1.1 cm.
> I think it is gone now. Since I used the wrong rx rotation twice, I think
> the two errors nearly, but not quite, cancelled each other. 
>   Of course, given the number of decimals used in GN 7.2 for the correct 
> result, there might still be a mismatch up to 0.001 arc seconds in long/lat 
> and up to 1 cm in height.