[postgis-users] issue with ST_Union on a large dataset

[adrian.kitchingman at dse.vic.gov.au]

Hi All

I'm having an issue with ST_Union on a large dataset. I have a LINESTRING 
layer which I want to union. The layer has been validated using ST_IsValid 
with no invalid geometries found. The LINESTRING layer is formed from a 
large MULTIPOLYGON layer which is also validated. The St_Union function 
works alright when operating on subsets of the data but throws a 
nondescript error (ERROR:  GEOS union() threw an error!) when run on the 
whole dataset. Are there known issues with ST_Union running on large 
datasets (~17000 geometries)? The SQL is below. I'm suspecting a memory 
issue but can't be sure. The same error has also occurred on our 64-bit 
Linux machine.

I've tried a number of work-arounds with no success. Firstly I tried 
breaking the process up so only spatially intersecting geometries were 
unioned and, secondly I've tweaked the processing memory variables for 
postgres. My current settings are below.

--works fine
CREATE TEMP TABLE all_lines AS
SELECT gid, st_boundary(geom) as geom
FROM (SELECT gid, (st_dump(geom)).geom as geom FROM featuresubset01) as 
foo 
GROUP BY gid, geom;

--throws and error when run on whole dataset
CREATE TEMP TABLE noded_lines AS
SELECT St_Union(geom) AS geom
FROM all_lines;

Postgres Memory Settings:
shared_buffers = 150MB
temp_buffers = 25MB
work_mem = 5MB
maintenance_work_mem = 16MB


Cheers

Adrian


Environ Specs:
Postgres 8.4.8
PostGIS 1.5.3
Windows XP 32-bit SP3
3.25GB RAM
Notice:
This email and any attachments may contain information that is personal, 
confidential, legally privileged and/or copyright.No part of it should be reproduced, 
adapted or communicated without the prior written consent of the copyright owner. 

It is the responsibility of the recipient to check for and remove viruses.
If you have received this email in error, please notify the sender by return email, delete 
it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information 
contained in this email.

Please consider the environment before printing this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20110801/c3b83ba7/attachment.html>