[postgis-devel] Infinite coordinates in geometries
strk
strk at keybit.net
Thu Jun 3 02:52:00 PDT 2004
I've added ! finite coordinates check in WKB parser as well..
Now it is *really* hard to detect other ! finite() related bugs.
A way could be making the GiST index routines warn in the presence
of non finite coordinates to try tracking where have them been
introduced. What do you think ? Is it worth it ?
As an alternative, we could have a check_for_infinite_geoms(schema,
table, column) func, checking for bounding box.
A simple check is actually checking extent(<column>) as any infinite
or NaN coordinate would make its extent infinite or NaN and this would
be the case for any union as well..
--strk;
On Thu, Jun 03, 2004 at 10:07:04AM +0200, strk wrote:
> On Wed, Jun 02, 2004 at 05:51:06PM -0700, Dave Blasby wrote:
> > Good points, strk.
> >
> > I dont think infinite geometries are valid (I'll check the spec tomorrow).
> > POINT(Inf Inf) isnt really defining where the point is in space.
> >
> > Inf coordinates are likely to screw up more than just the GiST index.
> >
> > My gut feeling is to make them invisible to the index (treat them like NULL),
> > and any query of the form:
> >
> > SELECT * FROM <table> && 'BOX3D(-Inf -Inf, ...)';
> >
> > to throw an error.
> >
> > Either that or throw an error whenever your geometry has an Inf or NaN in it.
>
> I've made WKT parser handle Inf coordinates in geom throwing an error
> and GiST ggeometry_compress skip Inf geoms as it does with EMPTY ones.
>
> WKB parser would still accept Inf I guess...
>
> >
> > What does JTS do with Inf coordinates? Is a geometry with Inf coordinates
> > valid (ie. geometry.isvalid() = true)?
>
> Dunno... can not make JTS parse an Inf geom from .xml ...
>
> --strk;
>
> >
> > Anyone else have suggestions?
> >
> > Quoting strk <strk at keybit.net>:
> >
> > >
> > > I've found the bug in the postgis picksplit function.
> > > This function is the exact copy of rtree_picksplit from rtree_gist,
> > > so this might be of interest to the rtree_gist guys as well.
> > >
> > > To determine destination of the node, this kind of checks are performed:
> > >
> > > cur->low.y - pageunion.low.y < pageunion.high.y - cur->high
> > > cur->low.y - pageunion.low.y > pageunion.high.y - cur->high
> > > cur->low.y - pageunion.low.y == pageunion.high.y - cur->high
> > >
> > > Substracting Inf from Inf gives Nan, which does not match any
> > > of the checks.
> > >
> > > I've currently (CVS) made the == check (equality) the default,
> > > so to have Infinite valued keys be cached by an ADD-TO-LESS-POPULATED
> > > rule.
> > >
> > > I dunno the consequences of this in terms of overall index functionality,
> > > but you don't get the error message anymore ;)
> > > I'm afraid Infinite geometry presence in a GiST index makes the index
> > > unusable because it eventualy ends up with an infinite plane as root
> > > node and with an arbitrary tree built under it.
> > >
> > > Maybe the picksplit function or whatever other GiST
> > > construction function invoked at every new inserted geometry or at
> > > while table scan should warn the user, or skip infinite geometries
> > > (making them invisible by the index).
> > >
> > > Any comment from GiST friends ?
> > >
> > > --strk;
> > >
> > > On Tue, Jun 01, 2004 at 10:14:55AM -0700, Paul Ramsey wrote:
> > > > Quoting strk <strk at keybit.net>:
> > > >
> > > > > On Tue, Jun 01, 2004 at 10:00:02AM -0700, Paul Ramsey wrote:
> > > > > > No, they shouldn't create Inf geometries.
> > > > > > It's 'POINT(Inf Inf)'::geometry, no comma, BTW.
> > > > > > P.
> > > > >
> > > > > Oops... it works, sorry :)
> > > > > Now, what should I do ?
> > > > > Drop Inf geometry support or use it to find other bugs ?
> > > >
> > > > I think we should drop it, what are your thoughts? If we leave it in, there
> > > is
> > > > an implied rational behavior to apply in the presence of these things which
> > > > could be very hard to figure out. I suppose from an indexing point of view
> > > they
> > > > are no worse than NULL geometries in semantic terms.
> > > >
> > > > >
> > > > > The operation our guy was trying to execute is not specified,
> > > > > I tried filling a table with Inf points, building a gist index,
> > > > > querying it... no memory alloc problems so far.
> > > > >
> > > > >
> > > > > --strk;
> > > > >
> > > > > >
> > > > > > Quoting strk <strk at keybit.net>:
> > > > > >
> > > > > > > On Tue, Jun 01, 2004 at 09:53:04AM -0700, Paul Ramsey wrote:
> > > > > > > > Intriguing... doing POINT(Inf,Inf) does not do the trick? I would
> > > have
> > > > > > > thought
> > > > > > > > that a2d() would interpret the Inf token properly.
> > > > > > > > P.
> > > > > > >
> > > > > > > test=# select 'POINT(Inf, Inf)'::geometry;
> > > > > > > ERROR: couldnt parse object in GEOMETRY
> > > > > > >
> > > > > > > If John Hagstrand could reproduce the Inf geoms that would help...
> > > > > > > I think the bug is actually there.. postgis functions should not
> > > > > > > produce such geoms, should 'hem ?
> > > > > > >
> > > > > > > --strk;
> > > > > > >
> > > > > > > >
> > > > > > > > Quoting strk <strk at keybit.net>:
> > > > > > > >
> > > > > > > > > On Tue, Jun 01, 2004 at 09:13:33AM -0700, Paul Ramsey wrote:
> > > > > > > > > > Sandro,
> > > > > > > > > > Could you check for a reasonable place to handle this special
> > > case?
> > > > > I
> > > > > > > do
> > > > > > > > > not
> > > > > > > > > > think Inf geometries are something we want to handle,
> > > necessarily
> > > > > :)
> > > > > > > > > > Paul
> > > > > > > > >
> > > > > > > > > The question is: how did this Inf geometry get created ?
> > > > > > > > > I dunno how to make one, for debugging purposes...
> > > > > > > > > --strk;
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ----- Forwarded message from
> > > john.hagstrand at interageresearch.com
> > > > > -----
> > > > > > > > > > Date: Tue, 1 Jun 2004 10:59:32 -0500
> > > > > > > > > > From: John Hagstrand <john.hagstrand at interageresearch.com>
> > > > > > > > > > Reply-To: John Hagstrand <john.hagstrand at interageresearch.com>
> > > > > > > > > > Subject: RE: [GENERAL] ERROR: invalid memory alloc request
> > > size 0
> > > > > > > > > > To: Paul Ramsey <pramsey at refractions.net>
> > > > > > > > > >
> > > > > > > > > > Hello Paul,
> > > > > > > > > >
> > > > > > > > > > I solved this problem.
> > > > > > > > > >
> > > > > > > > > > I had 20 or 30 records in that table with a geometry column
> > > like
> > > > > this:
> > > > > > > > > > POINT(-inf -inf). I deleted those records, and then dropped
> > > and
> > > > > > > recreated
> > > > > > > > > > the index. That solved the problem. I can now do inserts and
> > > > > updates
> > > > > > > > > > without getting the malloc error.
> > > > > > > > > >
> > > > > > > > > > Thanks for your help.
> > > > > > > > > > I look forward to seeing you in Ottawa next week.
> > > > > > > > > > John
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: Paul Ramsey [mailto:pramsey at refractions.net]
> > > > > > > > > > > Sent: Thursday, April 15, 2004 10:49 PM
> > > > > > > > > > > To: John Hagstrand
> > > > > > > > > > > Cc: Tom Lane; dblasby at refractions.net
> > > > > > > > > > > Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > > size 0
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > John,
> > > > > > > > > > > A pg_dump would be sufficient, or a shape file, and a SQL log
> > > of
> > > > > the
> > > > > > > > > > > commands you ran.
> > > > > > > > > > > Thanks!
> > > > > > > > > > > Paul
> > > > > > > > > > >
> > > > > > > > > > > On Thursday, April 15, 2004, at 12:27 AM, John Hagstrand
> > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hello Paul,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for helping. I'd like to send you the test data
> > > set.
> > > > > > > > > > > > What format would you like?
> > > > > > > > > > > >
> > > > > > > > > > > > John
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >> -----Original Message-----
> > > > > > > > > > > >> From: Paul Ramsey [mailto:pramsey at refractions.net]
> > > > > > > > > > > >> Sent: Wednesday, April 14, 2004 11:28 PM
> > > > > > > > > > > >> To: Tom Lane
> > > > > > > > > > > >> Cc: John Hagstrand; dblasby at refractions.net
> > > > > > > > > > > >> Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > > > > size 0
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >> We'll need the test data set that does this, of course,
> > > and
> > > > > some
> > > > > > > > > > > >> version info. The indexing routines seemed pretty solid
> > > > > (tested on
> > > > > > > > > > > >> many
> > > > > > > > > > > >> many millions of geometries), I am surprised this has come
> > > up.
> > > > > > > > > > > >>
> > > > > > > > > > > >> P.
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Wednesday, April 14, 2004, at 08:53 PM, Tom Lane wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>> "John Hagstrand" <john.hagstrand at interageresearch.com>
> > > > > writes:
> > > > > > > > > > > >>>> Here is a more detailed backtrace.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>> (gdb) bt
> > > > > > > > > > > >>>> #0 errfinish (dummy=0) at elog.c:319
> > > > > > > > > > > >>>> #1 0x081a85de in elog_finish (elevel=20,
> > > > > > > > > > > >>>> fmt=0x8231d80 "invalid memory alloc request size
> > > %lu")
> > > > > at
> > > > > > > > > > > >>>> elog.c:853
> > > > > > > > > > > >>>> #2 0x081b3dd6 in MemoryContextAlloc (context=0x9f10598,
> > > > > size=0)
> > > > > > > at
> > > > > > > > > > > >>>> mcxt.c:482
> > > > > > > > > > > >>>> #3 0x0807411d in gistSplit (r=0xbe9a1938, buffer=805,
> > > > > > > > > > > >>>> itup=0x9f25128,
> > > > > > > > > > > >>>> len=0xbfec3eac, giststate=0xbfec4070,
> > > res=0xbfec402c) at
> > > > > > > > > > > >>>> gist.c:1348
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Okay, this makes it pretty clear that the problem is the
> > > > > > > picksplit
> > > > > > > > > > > >>> function is setting v.spl_nleft to zero. In other words
> > > it's
> > > > > > > failed
> > > > > > > > > > > >>> to "split" the page at all (everything in the original
> > > index
> > > > > page
> > > > > > > has
> > > > > > > > > > > >>> gone into the righthand child). This seems to me to be
> > > > > > > necessarily
> > > > > > > > > > > >>> a bug in the index opclass's picksplit function.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> You previously said that the column in question is a
> > > PostGIS
> > > > > > > Geometry
> > > > > > > > > > > >>> column, so I'm of the opinion this is a PostGIS bug.
> > > (Paul,
> > > > > any
> > > > > > > > > > > >>> suggestions about how to probe deeper?)
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> regards, tom lane
> > > > > > > > > > > >>>
> > > > > > > > > > > >> Paul Ramsey
> > > > > > > > > > > >> Refractions Research
> > > > > > > > > > > >> Email: pramsey at refractions.net
> > > > > > > > > > > >> Phone: (250) 885-0632
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > Paul Ramsey
> > > > > > > > > > > Refractions Research
> > > > > > > > > > > Email: pramsey at refractions.net
> > > > > > > > > > > Phone: (250) 885-0632
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ----- End forwarded message -----
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > _______________________________________________
> > > postgis-devel mailing list
> > > postgis-devel at postgis.refractions.net
> > > http://postgis.refractions.net/mailman/listinfo/postgis-devel
> > >
> >
> >
> >
> >
> > _______________________________________________
> > postgis-devel mailing list
> > postgis-devel at postgis.refractions.net
> > http://postgis.refractions.net/mailman/listinfo/postgis-devel
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-devel
More information about the postgis-devel
mailing list