[postgis-devel] malloc/palloc/lwalloc

[Tom Lane]

"Paul Ramsey" <pramsey at cleverelephant.ca> writes:
> I've been trying to track down memory leaks in PostGIS code, and the
> power of palloc is getting in my way... since palloc handles all the
> memory cleanly, valgrind thinks that the code is leakless, which it
> is, from a system point-of-view, but it also, clearly leaks.

> Ordinarily, this might be fine ("so what, it leaks, it'll get cleaned
> up in the end") but in our case, we are leaking so *much* memory that
> the database actually OOMs on some big workloads. Hence the leak hunt,
> but a difficult hunt, without supporting tools.

> Any ideas on tools / tricks?

No magic bullets here I'm afraid.  A couple of ways I've tracked down
similar problems in the past:

* let the problem query run for awhile and then eyeball the contents of
the active palloc context to see what data there seems to be a lot of
copies of.  This works pretty well for string data but I suppose what
you're dealing with is mostly floats, which're going to be harder to
recognize by eye.

* build with -pg and use gprof to identify the biggest call sites of
MemoryContextAlloc.  This is far from perfect too, but it at least
lets you narrow things down.


It would be interesting to build some infrastructure to allow more
precise tracking of the sources of leaked palloc chunks, but I've
never been up against a problem that was hard enough to justify
taking the time to do that.

			regards, tom lane