[GRASS-dev] nasty export of doubles

Hamish hamish_b at yahoo.com
Wed Oct 8 00:57:21 EDT 2008


Hi,

this change
  http://trac.osgeo.org/grass/changeset/31236
introduced an unfortunate problem, whereas imported doubles have their
system level precision exposed (%.14f).

the general goal is that upon presentation by d.what.vect, v.db.select, or
whatever trailing 0s are nicely snipped away ala G_trim_decimal(); and
more over that in reasonable cases input == output. That's a reassuring
that the rest of the internals are ok, otherwise it is hard to spot when
something has gone wrong.

compare input:

 cat|station|sounder_m|lat|lon|easting|northing
 1|DC|94.3|-45°27.4492'|167°09.2427'|2053056.83|5511584.74
 2|CA|86.4|-45°24.2306'|166°59.0481'|2039364.05|5516567.25
 3|MR|407|-45°18.6105'|166°57.9204'|2037128.89|5526840.51
 4|BA|167.6|-45°33.5942'|166°54.6907'|2034985.01|5498850.4
 5|SC|168.2|-45°43.6784'|166°55.8538'|2037880.75|5480334.83
 6|LS|365.9|-45°59.0538'|166°48.5873'|2030638.43|5451227.37

with old lib/db/dbmi_base/valuefmt.c (%lf) output via v.db.select:

 cat|station|sounder_m|lat|lon|easting|northing
 1|DC|94.3|-45°27.4492'|167°09.2427'|2053056.83|5511584.74
 2|CA|86.4|-45°24.2306'|166°59.0481'|2039364.05|5516567.25
 3|MR|407|-45°18.6105'|166°57.9204'|2037128.89|5526840.51
 4|BA|167.6|-45°33.5942'|166°54.6907'|2034985.01|5498850.4
 5|SC|168.2|-45°43.6784'|166°55.8538'|2037880.75|5480334.83
 6|LS|365.9|-45°59.0538'|166°48.5873'|2030638.43|5451227.37

(they're identical)


and then with the current valuefmt.c (%.14f) output:

 cat|station|sounder_m|lat|lon|easting|northing
 1|DC|94.3|-45°27.4492'|167°09.2427'|2053056.83000000007451|5511584.74000000022352
 2|CA|86.40000000000001|-45°24.2306'|166°59.0481'|2039364.05000000004657|5516567.25
 3|MR|407|-45°18.6105'|166°57.9204'|2037128.88999999989755|5526840.50999999977648
 4|BA|167.59999999999999|-45°33.5942'|166°54.6907'|2034985.01000000000931|5498850.40000000037253
 5|SC|168.19999999999999|-45°43.6784'|166°55.8538'|2037880.75|5480334.83000000007451
 6|LS|365.89999999999998|-45°59.0538'|166°48.5873'|2030638.42999999993481|5451227.37000000011176


but if we use %lf, beyond %.6f would be rounded/truncated??
As demonstrated by northing 5498850.40000000037253 above, using %.14f
just guarantees 14 numbers after the ".", not add any more significant
digits.

so, given that projected coordinates in the range of millions, to 3
decimal places (mm), will be common -- how to tell the string to be
a max of 15 digits total, not assume some fixed number AFTER the "."
will cover it? is there some %15.15f or so?

In the past we tried %g there, but it was not suitable because the
easting/northing would jump into e+06, which is no good. (and
v.in.ascii created db tables will usually include easting and
northings)


I don't know if there is a "right" answer here, my vote would be to
go back to %lf as the lesser of the two evils.

again, this is about exposed numbers (to a DB or ascii), not
internally stored doubles for calculation, and I'd expect the
current %.14f to mislead some SQL (where FOO > 10.0) style query
sooner rather than later.


related; this change looks more reasonable:
  http://trac.osgeo.org/grass/changeset/25355

fwiw, %.8f should give about 1mm for lat/lon, which is beyond current
RTK GPS capability and what is meaningful for a lot of projection
math (as we are constantly reminded on the proj4 list).

while keeping in mind that people will press grass into non-geographic
tasks that we haven't considered, where those small ranges could be
useful. (although in those cases I'd hope they'd edit PROJ_UNITS to
use microns or whatever)


thanks,
Hamish


More information about the grass-dev mailing list