[postgis-devel] GeomUnion speedups

[Bill Binko]

On Sat, 25 Jun 2005 strk at refractions.net wrote:

... wonderful stuff about speedups elided...

That is wonderful news about the speedups.  I'm sorry I've been away, but 
we had a major floodding issue here yesterday, and we're finally back to 
normal.  I should have time to work on this tonight or tomorrow.

> 
> About memory we might implement a chunk-based incremental
> approach whereas blocks of geometries (defined by memory
> size maybe ?) would be buffer(0)'d to early dissolve 
> vertexes and reduce memory occupation. Thoughts on this ?

As I mentioned before, I really think the extra overhead incurred by 
MemGeomUnion (converting from PostGIS<->GEOS repeatedly) can be removed by 
passing an opaque handle to the internal GEOS structures that you're 
building.

That solves the problem when you're not dealing with 50 complex 
multi-polys (like your test), but with 330,000 simple polyes (like in 
mine).  If you load all of those into memory and detoast them all, you 
rapidly run into a memory issue, and start thrashing or failing.

Perhaps we could build do it in chunks (have the sfunc have a counter that 
trips a collect every X shapes) and have the ffunc do any that haven't 
been collected yet and then do the buffer()?

That would be nicely configurable (how many per chuck, etc).

I am surprised that the new collect/buffer is not impacted by order: is 
that a supposition?  Or did you test that?

Bill