[postgis-devel] brainstorming about topology polygonizer

[Sandro Santilli]

[sorry, previous mail sent too early]

On Fri, Sep 16, 2016 at 08:00:27AM +0200, Sandro Santilli wrote:

> It makes a total of 92815 SQL queries to be performed (rings x5).

[...]

> > I'll try a different approach, along these lines:
> > 
> >    geom = (GSERIALIZED *)PG_DETOAST_DATUM_COPY(dat);
> >    lwg = lwgeom_from_gserialized(geom);
> >    edge->geom = lwgeom_clone_deep(lwg);
> >    lwgeom_free(lwg);
> >    pfree(geom);
> > 
> > I'm afraid that doing so would still keep the Datum memory around
> > unless context memory is switched, which I suspect is not the case
> > as we call SPI_connect only once for the whole lifetime of the
> > function.
> 
> This test reduced the Maximum resident set size (kbytes) of the
> "Arezzo UCS" case from 
> 772400 to
> 722460
> 
> Not a huge benefit !

I finally found a bigger fish: the memory used to represent
the results of all those queries (92815 queries) were only released
at the very end of the function, even if not needed in between.

Basically my suspect was confirmed, of memory being around till
SPI_finish. Explicitly releasing SPI_tuptable when not needed anymore
brings down the Maximum resident set size (kbytes) from
722460 to
167588

Sounds better, doesn't it ? 
That's 167 MB.
Still a bit high wrt the 13 MB needed to hold
in memory the geometries of all the faces, but
we're talking about the resident set size for the whole PostgreSQL
process, which is configured to use up to 128 MB for work_mem ...

I've committed the early memory release in trunk, even if I don't
think there's in trunk any topology function implemented in C
and running so many queries.

Thanks for listening :)

--strk;