[postgis-users] snapToGrid strange behaviour

[Michael Fuhr]

On Fri, Jan 27, 2006 at 08:09:37AM -0800, Kevin Neufeld wrote:
> That's exactly what I was thinking as well. But the question remains, 
> why does snapToGrid(...) not actually snap to 0.001 as instructed?

Presumably it does, but wee differences are probably creeping in
because 0.001 and the resulting values don't have exact binary
representations.  Those errors are smaller than the double precision
type's stated accuracy (15 decimal digits or so), but they make the
values different when doing bit-for-bit equality tests.  So if the
"same" value is arrived at by different calculations, the binary
representations might be different.  Example:

test=> CREATE TABLE foo (d1 float8, d2 float8, d3 float8, d4 float8, d5 float8);
test=> INSERT INTO foo VALUES (1.0, 1000.0, 999.0);
test=> UPDATE foo SET d4 = d1 / d2, d5 = d1 - d3 / d2;
test=> SELECT d4, d5, d4 = d5 FROM foo;
  d4   |  d5   | ?column? 
-------+-------+----------
 0.001 | 0.001 | f
(1 row)

test=> SELECT encode(float8send(d4), 'hex'), encode(float8send(d5), 'hex')
test-> FROM foo;
      encode      |      encode      
------------------+------------------
 3f50624dd2f1a9fc | 3f50624dd2f1aa00
(1 row)

-- 
Michael Fuhr