[postgis-users] snapToGrid strange behaviour

[Kevin Neufeld]

That's exactly what I was thinking as well. But the question remains, 
why does snapToGrid(...) not actually snap to 0.001 as instructed?


Michael Fuhr wrote:

>On Thu, Jan 26, 2006 at 02:25:58PM -0800, Kevin Neufeld wrote:
>  
>
>>Can somebody please explain why we are getting these results?
>>    
>>
>[...]
>  
>
>>-[ RECORD 1 ]----------------------------------------
>>geom     | 0101000000C3F5285C8FC2F33F1B2FDD2406C12340
>>geom     | 0101000000C2F5285C8FC2F33F1A2FDD2406C12340
>>astext   | POINT(1.235 9.877)
>>astext   | POINT(1.235 9.877)
>>equals   | f
>>?column? | f
>>    
>>
>
>I think you're encountering the usual inaccuracies of representing
>floating-point numbers in binary.  Notice the difference in the
>internal representations of the two geometries: in the first the
>y-coordinate is 1B2FDD2406C12340; in the second it's 1A2FDD2406C12340.
>Since the data is little-endian (as indicated by the leading 01),
>the values are actually 0x4023C10624DD2F1B and 0x4023C10624DD2F1A,
>which differ only in the lowest bit.  So instead of being exactly
>9.877 the values are more like 9.8770000000000007 and 9.8769999999999989.
>The calculations that yielded those numbers must have been slightly
>different, resulting in that one-bit discrepancy.
>
>  
>

-- 
Kevin Neufeld,
Refractions Research Inc.,
kneufeld at refractions.net
Phone: (250) 383-3022 
Fax:   (250) 383-2140