[postgis-devel] malloc/palloc/lwalloc

[Paul Ramsey]

Tom,

Since I have you here...

I've been trying to track down memory leaks in PostGIS code, and the
power of palloc is getting in my way... since palloc handles all the
memory cleanly, valgrind thinks that the code is leakless, which it
is, from a system point-of-view, but it also, clearly leaks.

Ordinarily, this might be fine ("so what, it leaks, it'll get cleaned
up in the end") but in our case, we are leaking so *much* memory that
the database actually OOMs on some big workloads. Hence the leak hunt,
but a difficult hunt, without supporting tools.

Any ideas on tools / tricks?

P.

On Tue, Sep 23, 2008 at 8:40 PM, Tom Lane <tgl at sss.pgh.pa.us> wrote:
> "Paul Ramsey" <pramsey at cleverelephant.ca> writes:
>> Makes sense. Grepping finds a few bare mallocs in there. Also lots of
>> lwalloc lwfree in the lwgeom area. Does the idea of a bare malloc make
>> any extra sense taken in combination with the fact that GEOS is going
>> to be doing bare malloc all over the place anyways? Or
>> memory-is-memory-is-memory?
>
> Not familiar at all with the GEOS or Postgis code, just Postgres;
> so take this advice for what its worth...
>
> I think the $64 question is how your memory allocation and error
> handling strategies interact.  If you have code that throws errors
> using PG's elog() or ereport(), then you'd better be allocating memory
> with palloc, because you won't get a chance to clean it up yourself.
> Consider
>
>        mymem = palloc(...);
>        ...
>        if (trouble)
>                elog(ERROR, "trouble");
>        ...
>        pfree(mymem);
>
> If the elog is reached then the pfree won't be, and so you need the
> behind-the-scenes machinations of palloc/pfree to clean up.  If you
> used plain malloc/free for mymem you'd have a memory leak here.
> (Yeah, in this example you could put a free() just before the elog(),
> but throw a few levels of subroutine calls into the example and it
> gets unworkable PDQ.)
>
> However, non-Postgres-oriented code like GEOS is most likely designed
> to report errors via error return codes and to clean up its allocations
> on the way out.  That works fine too, though it requires plenty of
> attention to detail to ensure every failure path releases what it
> should.
>
> Where things fall down badly is where you mix the two styles.  In
> particular, don't throw an elog/ereport from inside code called by
> GEOS.
>
>                        regards, tom lane
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-devel
>