[postgis-devel] BBOX caches: float or double ?

[Sandro Santilli]

On Wed, Nov 09, 2011 at 07:43:21PM +0100, Sandro Santilli wrote:
> On Wed, Nov 09, 2011 at 09:39:38AM -0800, Paul Ramsey wrote:
> > Where in g_serialized.txt do you see the suggestion that the
> > serialized boxes are doubles? That is not the intent, so I am happy to
> > clarify: that doesn't happen. The serialization stores the boxes as
> > floats.
> 
> Oh, I was wrong, sorry. <> is float, [] is double. Docs are ok.
> 
> > Now, when *deserialized* into LWGEOM->GBOX, they are held in doubles,
> > but that shouldn't be a problem, IMO.
> 
> I agree that's not a problem. It is a problem computing it and _not_
> rounding it to float though, if we'd have the values rounded to float
> when found in the cache (which is the problem with #1273).

I'm attaching a patch for the specific case of #1273, but a review
would be needed to ensure that _any_ BBOX computation would end up
being rounded to nearest higher/lower value, using the next_float_down
and next_float_up functions.  It could help to have a void
round_to_float(GBOX*) function.

Also, we need many more tests in the testcase about this float<->double
thingy.

--strk;

  ()   Free GIS & Flash consultant/developer
  /\   http://strk.keybit.net/services.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: next_float.patch
Type: text/x-diff
Size: 970 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20111110/70040002/attachment.patch>