[postgis-devel] [postgis-users] Memory management in postgres (with liblwgeom functions in particular)

[Paul Ramsey]

Igor,

WRT to using liblwgeom for gserialize support, depending on your actual needs you might want to look at the approach taken by https://github.com/pgpointcloud for PostGIS support. Rather than having a hard dependency on liblwgeom, use a soft dependency on the postgis extension itself, and pass the data through the pgsql bytea type using WKB as the transport. This does involve having a wkb reader/writer in your code, but it will be guaranteed to be less brittle over time and reduce your extension having dependencies on particular versions of liblwgeom.

Can you share what it is you’re planning to build that is both “postgis” and “not postgis” at the same time?

P. 


--  
http://postgis.net  
http://cleverelephant.ca


On March 30, 2015 at 8:20:46 AM, Paul Ramsey (pramsey at cleverelephant.ca) wrote:
> Igor,
> Your supposition is is all correct (you might want to ask your
> questions on postgis-devel, where the developer density is higher).
> lwalloc by default is just a facade over malloc, for standalone use.
> But, when used in PostGIS, it it backed by palloc, which is in turn
> the memory manager provided by PgSQL. The PgSQL memory manager is a
> heirarchical memory manager that provides indepedent blocks of memory
> for different levels of operation. So a SQL query will get a block,
> and every palloc in that context will go into that block. Then when
> the query is done, the whole block gets thrown away. The upshot is
> that code that uses palloc in relatively short-lived contexts (like
> queries) can be pretty lazy about memory management, since the whole
> context is getting tossed at the end anyways.
> P.
>  
> On Mon, Mar 30, 2015 at 8:11 AM, Igor Stassiy wrote:
> > Hello,
> >
> > I am developing a C++ extension (most of the code is C++) for postgres that
> > links dynamically with liblwgeom, without linking to postgis. I call
> > liblwgeom functions that serialize/deserialize LWGEOM* (and similar
> > structures) that don't need a backend like GEOS.
> >
> > I wonder how is the memory freed when we call lwerror, as the transaction
> > will be terminated (elog/ereport create a long jump), so if I call
> > liblwgeoms functions from within C++, the stack will not be unwind and even
> > if I use smart pointers it wouldn't make a difference (right?).
> >
> > On the contrary, when Postgis module loads itself, in _PG_init it overrides
> > memory allocation functions of liblwgeom with pg_alloc and pg_free. Which in
> > turn call palloc and pfree. And in this case when we call lwerror, the
> > memory that we allocated is freed automatically (right?).
> >
> > I guess (just a guess) it has something to do with the memory context and
> > when a memory context is "closed" the entire memory allocated within would
> > be freed. But lwalloc by default is malloc, so does Postgres do something
> > extremely clever like overriding malloc with its palloc?
> >
> > Thank you,
> > Igor
> >
> > _______________________________________________
> > postgis-users mailing list
> > postgis-users at lists.osgeo.org
> > http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>