[postgis-users] ERROR: array size exceeds themaximumallowed(134217727)

[Paul Ramsey]

Mike,
See if this patch makes anything better. It matches tlane's changes to
the 8.4 agg finalfunc.
P.

On Mon, Mar 22, 2010 at 1:29 PM, Greg Stark <gsstark at mit.edu> wrote:
> On Mon, Mar 22, 2010 at 4:45 PM, Paul Ramsey <pramsey at cleverelephant.ca> wrote:
>> Did you already try replacing your postgis functions with array_agg
>> calls to see if we can push the problem back over the fence to pgsql
>> land?
>
> On Sat, Mar 20, 2010 at 9:17 AM, Mike Leahy <mgleahy at alumni.uwaterloo.ca> wrote:
>> Running this query on various data will produce one of two results.  One is
>> the error mentioned in the subject (ERROR:  array size exceeds the maximum
>> allowed (134217727)).  I can find very little information on this error.  The
>> other outcome is that it often causes the PostgreSQL backend to segfault (see
>> gdboutput.txt).
>
> gdboutput.txt:
>
> Program terminated with signal 11, Segmentation fault.
> #0  0x00007fa4be23615b in pfree () from
> /usr/lib/postgresql/8.4/bin/postgres
> #1  0x00007fa4be18091b in makeMdArrayResult () from
> /usr/lib/postgresql/8.4/bin/postgres
> #2  0x00007fa4b7f038bc in pgis_accum_finalfn () from
> /usr/lib/postgresql/8.4/lib/postgis-1.5.so
> #3  0x00007fa4b7f039ee in pgis_geometry_union_finalfn () from
> /usr/lib/postgresql/8.4/lib/postgis-1.5.so
>
> Any chance you can generate one of these seg faults from a build with
> symbols and with assertions enabled? It might catch the problem
> earlier and provide more info.
>
> IIRC there were some memory management changes which required changes
> to array_agg() and which had some risk of causing problems for other
> sites with similar coding. Is it possible your'e missing some of these
> changes? I'm having trouble tracking them all down but at least
> there's these:
>
>
> commit 3d332de2eab8a01c0ef3f58ea69de2010fe8a1e1
> Author: Tom Lane <tgl at sss.pgh.pa.us>
> Date:   Thu Jul 23 20:45:27 2009 +0000
>
>    In a non-hashed Agg node, reset the "aggcontext" at group
> boundaries, instead
>    of individually pfree'ing pass-by-reference transition values.  This should
>    be at least as fast as the prior coding, and it has the major advantage of
>    clearing out any working data an aggregate function may have stored in or
>    underneath the aggcontext.  This avoids memory leakage when an aggregate
>    such as array_agg() is used in GROUP BY mode.  Per report from Chris Spotts.
>
>    Back-patch to 8.4.  In principle the problem could arise in prior versions,
>    but since they didn't have array_agg the issue seems not critical.
>
> commit 7f83b61cc26eeac0c5a09add49f6cf899f87fc0b
> Author: Tom Lane <tgl at sss.pgh.pa.us>
> Date:   Sat Jun 20 18:45:28 2009 +0000
>
>    Fix things so that array_agg_finalfn does not modify or free its input
>    ArrayBuildState, per trouble report from Merlin Moncure.  By adopting
>    this fix, we are essentially deciding that aggregate final-functions
>    should not modify their inputs ever.  Adjust documentation and comments
>    to match that conclusion.
>
> --
> greg
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: finalfunc.patch
Type: application/octet-stream
Size: 868 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20100322/9e175b4b/attachment.obj>