[postgis-devel] BBOX caches: float or double ?

[Nicklas Avén]

Hallo

An example when ordering by <-> not orders by distance to the actual
points:

SELECT id, ST_Distance(a, b), a<->b FROM 
(SELECT 1 id, 'POINT(300000 6000000)'::geometry a, 'POINT(300001.0
6000001.4)'::geometry b 
UNION ALL
SELECT 2 id, 'POINT(300000 6000000)'::geometry a, 'POINT(300001.1
6000001.1)'::geometry b ) c
ORDER BY  a<->b

The query returns the two rows in wrong order with respect to the
st_distance result. As you can see from the results the difference
between <-> and st_difference is about 3 dm in this example. But that is
the reality around here when using utm33. Almost all digits in a float
is used just for the integer part of the coordinate.


/Nicklas



On Thu, 2011-11-10 at 11:05 +0100, Sandro Santilli wrote:
> On Wed, Nov 09, 2011 at 07:43:21PM +0100, Sandro Santilli wrote:
> > On Wed, Nov 09, 2011 at 09:39:38AM -0800, Paul Ramsey wrote:
> > > Where in g_serialized.txt do you see the suggestion that the
> > > serialized boxes are doubles? That is not the intent, so I am happy to
> > > clarify: that doesn't happen. The serialization stores the boxes as
> > > floats.
> > 
> > Oh, I was wrong, sorry. <> is float, [] is double. Docs are ok.
> > 
> > > Now, when *deserialized* into LWGEOM->GBOX, they are held in doubles,
> > > but that shouldn't be a problem, IMO.
> > 
> > I agree that's not a problem. It is a problem computing it and _not_
> > rounding it to float though, if we'd have the values rounded to float
> > when found in the cache (which is the problem with #1273).
> 
> I'm attaching a patch for the specific case of #1273, but a review
> would be needed to ensure that _any_ BBOX computation would end up
> being rounded to nearest higher/lower value, using the next_float_down
> and next_float_up functions.  It could help to have a void
> round_to_float(GBOX*) function.
> 
> Also, we need many more tests in the testcase about this float<->double
> thingy.
> 
> --strk;
> 
>   ()   Free GIS & Flash consultant/developer
>   /\   http://strk.keybit.net/services.html
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-devel