[GRASS5] Re: [GRASS_DE] r.mapcalc round()

[Roger Bivand]

On Fri, 1 Jun 2001, Glynn Clements wrote:

> 
> > where the underlying C code has to take care of machine dependencies as
> > well as retaining default values for the number of digits to round to, and
> > the number of significant digits to retain. The resulting returned values
> > are then handed off to formatting code. One reason for the difficulties is
> > that different users may need very different precisions for different
> > purposes, and this would in our case add to the arguments taken by
> > programs outputting FCELL and DCELL as text.
> 
> Yep; so what should the syntax of these arguments be?
> 
> The simplest option (to implement) would be to allow a literal format
> string to be specified, e.g. "fmt=%.6g". But that isn't necessarily
> the best option for users.
> 
...
> 
> Thinking about this a bit, there are quite a few different ways to
> specify the precision, including (but probably not limited to):
> 
> 	Number of decimal places
> 	Number of significant figures
> 	Relative error
> 	Absolute error
> 
It looks as though number of significant figures is the most important,
because that takes care of relative error too. Absolute error is maybe too
scale-dependent to help, I don't have a feel for that unless that's what a
client asks about? Number of decimal places is also dependent on where the
point is to start with, so significant digits seem to be the way to go,
and the line of least resistance is then %<sig.digs>e, (taking into
account platform dependencies with regard to the exponent). Default values
would follow from machine representation, for DCELL say 16 and for FCELL
say 8, but stored as environment variables.

I'm only too clear about this still jumping over Markus' original
question, which has at least as much to do with the apparent oddness of
numbers being different between input and output when they appear with
"too many" significant digits.

Roger


-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand at nhh.no
and: Department of Geography and Regional Development, University of
Gdansk, al. Mar. J. Pilsudskiego 46, PL-81 378 Gdynia, Poland.