[postgis-devel] error with GIST indexes with NULLs

[strk at refractions.net]

On Fri, Dec 15, 2006 at 05:46:28PM +0000, Mark Cave-Ayland wrote:
> On Fri, 2006-12-15 at 17:00 +0100, strk at refractions.net wrote:
> 
> Hi strk,
> 
> > On Fri, Dec 15, 2006 at 07:36:09AM +0000, Mark Cave-Ayland wrote:
> >
> > > Confirmed. I've had a quick look at this with a debugger, and it appears
> > > that the culprit is BOX2D_union calling PG_RETURN_NULL, which causes
> > > DirectFunctionCall2 to error out in fmgr.c.
> > 
> > BOX2D_union is likely *always* returning NULL. We're talking about 459
> > success and 1 failure ...
> 
> AIUI the GiST penalty clause is called to determine into which subtree
> it is the least expensive to insert into - so my guess is that the
> insertion is falling over just after the first page split, but I don't
> have enough time to investigate at the moment :(

I don't have 8.2 around, anyway the attached patch avoids calling DirectFunctionCall2
if both inputs are NULL. Kevin, can you check if it fixes your issue ?
Also, could you add a testcase for this in the SVN repository ? We already have
a regress_lots_of_points.sql file, we should add a regress_lots_of_null.sql and
use it in a similar way from regress_index.sql.

--strk;
-------------- next part --------------
Index: lwgeom/lwgeom_gist.c
===================================================================
--- lwgeom/lwgeom_gist.c	(revision 2557)
+++ lwgeom/lwgeom_gist.c	(working copy)
@@ -959,14 +959,25 @@
 #endif
 
 
-	ud = DirectFunctionCall2(BOX2D_union, origentry->key, newentry->key);
-	/*ud = BOX2D_union(origentry->key, newentry->key); */
-	tmp1 = size_box2d_double(ud);
-	if (DatumGetPointer(ud) != NULL)
-		pfree(DatumGetPointer(ud));
+	if (DatumGetPointer(origentry->key) == NULL && DatumGetPointer(newentry->key) == NULL)
+	{
+#ifdef PGIS_DEBUG_GIST6
+		elog(NOTICE,"GIST: LWGEOM_gist_penalty called with both inputs NULL");
+#endif
+		*result = 0;
+	}
+	else
+	{
 
-	*result = tmp1 - size_box2d_double(origentry->key);
+		ud = DirectFunctionCall2(BOX2D_union, origentry->key, newentry->key);
+		/*ud = BOX2D_union(origentry->key, newentry->key); */
+		tmp1 = size_box2d_double(ud);
+		if (DatumGetPointer(ud) != NULL)
+			pfree(DatumGetPointer(ud));
 
+		*result = tmp1 - size_box2d_double(origentry->key);
+	}
+
 #ifdef PGIS_DEBUG_GIST6
 	elog(NOTICE,"GIST: LWGEOM_gist_penalty called and returning %.15g", *result);
 #endif