[postgis-devel] Bugs in gserialized_gist_compress_2d

[Paul Ramsey]

I don't really see how the logic in gserialized_gist_compress_2d
differs from that in the core PgSQL code, like say gist_poly_compress.
In the event of things going terribly wrong, it's returning the
unaltered input.

P.

On Wed, Sep 11, 2013 at 10:56 AM, Paul Ramsey <pramsey at cleverelephant.ca> wrote:
> Thoughts on what we *should* do with invalid inputs? Have invalid keys? Not sure what the implications of a nan key, or inf key are.
> P.
>
> --
> Paul Ramsey
> http://cleverelephant.ca/
> http://postgis.net/
>
>
> On Sunday, 8 September, 2013 at 3:23 AM, Alexander Korotkov wrote:
>
>> Hi!
>>
>> While investigating NaN handling in GiST indexes I found two buggy checks gserialized_gist_compress_2d:
>>
>> /* Is the bounding box valid (non-empty, non-infinite)? If not, return input uncompressed. */
>> if ( result == LW_FAILURE )
>> {
>> POSTGIS_DEBUG(4, "[GIST] empty geometry!");
>> PG_RETURN_POINTER(entry_in);
>> }
>>
>>
>> /* Check all the dimensions for finite values */
>> if ( ! finite(bbox_out.xmax) || ! finite(bbox_out.xmin) ||
>> ! finite(bbox_out.ymax) || ! finite(bbox_out.ymin) )
>> {
>> POSTGIS_DEBUG(4, "[GIST] infinite geometry!");
>> PG_RETURN_POINTER(entry_in);
>> }
>>
>>
>> On failure these checks return original GISTENTRY with geometry key. I don't understand what is it supposed to do, but what it's actually do is to interpret first 16 bytes of geometry as box3df. For sure, it leads to broken index key and invalid answers from index scan.
>> ------
>> With best regards,
>> Alexander Korotkov.
>> _______________________________________________
>> postgis-devel mailing list
>> postgis-devel at lists.osgeo.org (mailto:postgis-devel at lists.osgeo.org)
>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel
>
>
>