[OSRS-PROJ] Significant digits in parameters

[Craig Bruce]

Gerald Evenden <gerald.evenden at verizon.net> wrote:

> I presume you mean that prog A produces the same results as prog B
> to 15+ digits.  I am sorry but I couldn't disagree with you more for
> several reasons:
> 1) it would make an observer of the tabulation of results assume that
> these values were accurate to the printed significance.

People won't normally see these figures.  When displaying results for
people to read, you should round and convert the figures into whatever
display format is expected, like -75°12'03.12".

> 2)different programs that are perfectly usable in the case of say 1mm
> precision will create chaos when compared with another program accurate to
> 1nm precision.  Which program is correct?  Which program do I throw
> out?

A similar effect is achieved by programs the produce inconsistent results.
Where things can be controlled, we should make efforts to insure that
programs produce the same results wherever we can.  And this effort is
very easy in this case.

> I am not sure what is going on in the following paragraph but I must
> say that the discussed method has a serious defect if it is dependent
> upon knowing the precision of the input but has no way of having
> this condition as controlling input and merely depends upon the
> precision of double precision.

Software developer do not always have complete control over the software
libraries they are using.  Error control is applied in good programming
to control at least double-precision errors of fully precise analytical
results.  I.e., lower-level software may assume that the figures provided
to it are accurate to 13 digits because it has no idea that it is dealing
with geographic figures; it's just general purpose.

For instance, if you were to take 10° longitude and transform it to
a projection and then back again, you might get 9.999999999999713°
instead of 10°.  A clever program that, say, plots a pixel for this
onto the screen will have at least some error control to make figures
that are very close to 10° be plotted into the 10° location (say that
this was on a pixel boundary, which is not uncommon for round figures).
If some software library arbitrarily chose the rounding to be 13 digits,
because it is general-purpose and calculation errors normally only consume
the last two or three digits of a 'double', then it would do the right
thing in this case.

However, if the 10° figure was transformed by one program that
rounded off projection-parameter definitions arbitrarily and then the
result was inverted by some other program, the output figure might be
9.999999999178631° in double precision, and the plotting program that
only compensates for regular double-precision errors will plot the pixel
into the wrong place.  When images from multiple sources are overlayed,
this difference may look ugly.

Go and ask Microsoft, for example, how they control floating-point errors
when plotting things or tell them to add a parameter to the plotting
interface to give the precision of the coordinate figures and watch how
quickly they don't snap to attention.

--------------------------+------------------------+--------------------------
Dr. Craig S. Bruce        | Tel: 819-771-8303 x205 |             CubeWerx Inc.
Senior Software Developer |   Home: 613-565-1344   |  Gatineau, Québec, Canada
csbruce at cubewerx.com      | http://www.csbruce.com |  http://www.cubewerx.com/
--------------------------+------------------------+--------------------------
    "Everything should be built from the top down, except the first time."
----------------------------------------
PROJ.4 Discussion List
See http://www.remotesensing.org/proj for subscription, unsubscription
and other information.