[postgis-devel] Status of custom memory allocators based interruptable calls

[Sandro Santilli]

On Thu, May 24, 2012 at 10:37:10AM -0700, Chris Hodgson wrote:
> It seems like a pretty big wall to put up: "If you want to be used
> by PostGIS, you must use memory in a supported way and/or provide
> for external memory management" - especially since it is PostGIS
> that wants to use the library, not the other way around.

It's actually different. My finding is you don't need to do anything
special to "provide for external memory management". Any C++ code can
_force_ linked libraries to use its own memory managent functions.

This is what happened with GDAL. I didn't mean to have it use our
allocators (and I'll likely work for avoiding that).

> It seems like the goal of interruptible calls may suffer from the
> same problem, worded slightly differently: "If you want to be used
> by postgis, you must support a way to interrupt long-running
> processes". Besides just the interruptibility, which we could inject
> using the overridden new/delete approach, we also need to allow for
> memory cleanup... so it seems to further compound the problem.

Yep, on idea was to pick both birds with the same stone, with custom
allocators faking an out of memory condition on "interrupt", which
supposedly any sane library is handling cleanly already.
It turns out GDAL doesn't actually do so, possibly aborting the whole
process on out-of-memory ...

> How do other PostgreSQL extensions deal with this? There must be
> some XML database extension that uses an external (possibly C++)
> library? Do they actually allow for interrupting long running XML
> parsing, etc? Either we're in uncharted waters, in which case I
> suggest you get back in the boat and motor on... or there is some
> example out there of how to do this. Sorry but I don't have one
> off-hand.

Nothing came out on pgsql-hackers:
http://archives.postgresql.org/pgsql-hackers/2012-05/msg00736.php

I've been thinking I could just try to handle SIGINT in GEOS
to cleanly interrupt whatever operation it is doing. I'll try
this out today.

> I appreciate the effort you're putting into this, Good Luck!

Thanks for the comfort :)

--strk;