[GRASS5] shift bug in r.in.ascii/r.out.ascii

[Roger Bivand]

On Fri, 6 Jul 2001, Helena wrote:

> Markus Neteler wrote:
> 
> > I have just run the same test - it seems to be working alright.
> 
> thanks for testing it
> 
> >
> > But... even here we have the precision problem:
> >
> > ncols 140
> > nrows 144
> > xllcorner 3568200
> > yllcorner 5765462.5
> > cellsize 12.500000
> > NODATA_value -9999
> > 126.870003 126.550003 126.25 125.93 125.599998
> >        ^^^       ^^^^                    ^^^^^
> 
> Markus, the numbers are perfectly right for
> floating point, if you want to have 9 digits (126.870000) then you would have
> to store that number as double, which of course can be done but as we discussed
> before do you want to have your milion cell DEM with vertical accuracy in dm stored
> as doubles? I don't have a problem with this because this is a good reminder for me
> how the data are stored and that it needs to be taken into account when running
> the models. If it bothers you - change the output to just 7 digits. So for example if
> you are using
> r.out.ascii for floating point raster map and your elevations are say 100.000000 -
> 150.000000
> you should use dp = 4 rather than the default 6, because the FP holds only 7 digits
> (correct me somebody if I am not right). If you want those data output with 6 decimals
> places
> you would need double precission.
> r.stats does not give you that choice so it may be useful to add it there and then there
> are also
> some programs which print out numbers with ridiculous number of digits, e.g. r.colors:
> I counted 29 digits for r.colors data range when you run it with the rules option . That
> should be
> easy to fix.
> As it was already discussed it would be possible to have grass automatically adjust
> everything
> related to fp to just 7 digits but there may be cases when this may be too restrictive.
> Because nothing is really broken here maybe the best approach would be that whoever is
> working
> on a program where dp option should be added or number of digits for print out needs to
> be adjusted
> fixes that too, or when time allows it along with other more important things.
> 
> I guess that I have written more than I really wanted to,
> 

Not at all! Some of these representation things need to be repeated when
the time is right - that is when people see the trailing digits as
representing something. FP gives you - as you say - about 7 digits, which
is fairly precise for continuous phenomena, DP will give you about 16. FP
loses out numerically when you do things to it, like lots of trigonometry,
and the number of digits you can trust will rot from the right - this
isn't COBOL or database stuff! The binary representation _is_ just a
representation, in DP does a couple of hectares more or less matter at
continental scales?

Roger

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand at nhh.no
and: Department of Geography and Regional Development, University of
Gdansk, al. Mar. J. Pilsudskiego 46, PL-81 378 Gdynia, Poland.