[postgis-users] Hints or tips on Large Intersect

[Ben Jubb]

Hi Niels,
It seems like PostGIS didn't build against the correct version of GEOS. 
Can you verify that the PostGIS is including the correct geos_c.h?

I've checked in a revised version of the PostGIS code.  I made a change 
to improve performance, at least for large base geometry cases.  The SQL 
interface now has overloaded versions of ST_intersects (contains, 
containsProperly, and covers) that take an additional integer argument.  
This 3rd argument is a surrogate key whose value is used to determine 
when the 1st geometry argument has changed.  So you need to arrange that 
this value changes whenever the first geometry changes, or just set it 
to constant value (like 0) if you know the first geometry argument will 
never change during the execution of the query.  If don't pass this 3rd 
argument, you get the normal behaviour of ST_intersects.

b

niels hoffmann wrote:
> I installed both the development versions of postgis and geos on a Fedora Core5 test box running PostgreSQL 8.1.3
> "POSTGIS=""1.3.3SVN"" GEOS=""3.1.0-CAPI-1.5.0"" PROJ=""Rel. 4.6.0, 21 Dec 2007"" USE_STATS"
> When I try to execute the intersect query with the ST_IntersectsPrepared I get the error: 
> SELECT ST_intersection(a.geom, l.geom) as intersect_geom, a.*, 
> l.class,l.name <http://l.name>, l.replid from first_table l, 
> second_table a Where ST_IntersectsPrepared(a.geom, l.geom);
>  
> ERROR:  Not implemented in this version!
>  
> The original query:
> SELECT intersection(a.geom, l.geom) as intersect_geom, a.*, 
> l.class,l.name <http://l.name>, l.replid from first_table l, 
> second_table a Where a.geom && l.geom AND intersects(a.geom, l.geom);
>  
> Throws an error as well:
>  
> NOTICE:  IllegalArgumentException: Exponent out of bounds
>  
> ERROR:  GEOS Intersection() threw an error!
> Does anybody have a suggestion how to troubleshoot this?
> I grabbed GEOS from the SVN repository, the hourly svn snapshot link 
> on the geos page seems to give me the 3.0.0rc5 version?
>  
> Cheers, Niels
>  
> Ben Jubb wrote: 
> If you are comfortable with building your own version of PostGIS, you
> could get the latest development version from the SVN repository (you'll
> need to get the latest dev version of GEOS too),  and try using the new
> 'prepared' predicate ST_IntersectsPrepared.  This is a replacement for
> the intersects() function in your code, that (depending on the
> geometries involved) could speed up that test by 100s of times.  It
> optimized for the case when the first argument is changing slowly, and
> the second argument is changing every invocation.
>  
> What do you think is bottleneck in your code though?  Is it the
> intersection() operation or the intersects() test?  It depends on the
> distribution of your data, and the odds of any pair of geometries
> intersecting.
>  
> b
>  
> Stephen Woodbridge wrote:
> > > Yeah, this sounds like it will run for a very very long time. A 
> couple
> > > of thoughts that have more to do with managing the process:
> > >
> > > 1) make sure you run an is_valid() check on both tables and 
> remove/fix
> > > any geometries that are not valid. It is a pain when you hit one of
> > > these and it nukes your transaction or crashes the server and you 
> have
> > > to start over.
> > >
> > > 2) you might want to break this into multiple queries based on some
> > > subset of the record in the smaller table. Like do 1-10000,
> > > 10001-20000, etc. This would allow you to get the results of each
> > > commited so a restart would be less painful, also this would allow 
> you
> > > to get some timing statistics to better predict how long the 
> remainder
> > > of the rows will take.
> > >
> > > -Steve
> > >
> > > niels hoffmann wrote:
> >> >> Hi,
> >> >> 
> >> >> I am fairly new to Postgis so I like some feedback whether I am 
> going
> >> >> through the right moves.
> >> >> I am trying to create a new table with the intersected results from
> >> >> two input polygon tables.
> >> >> Both tables are in NZMG (2193) the first table has 100000+ records,
> >> >> the second table has 400000+ records.
> >> >> The query I am using is:
> >> >> 
> >> >> Create table ablc_pol with OIDS as
> >> >> SELECT intersection(a.geom, l.geom) as intersect_geom, a.*,
> >> >> l."CLASS",l."NAME", l."REPLID"
> >> >> from first_table a, second_table l
> >> >> Where a.geom && l.geom
> >> >> AND intersects(a.geom, l.geom);
> >> >> 
> >> >> Currently this query is taking >200 hours before I cancelled it
> >> >> because I wasn't sure it would ever end. However, running it on a
> >> >> small subset showed satisfactory results...
> >> >> I am using version 1.2 on Windows.
> >> >> Does it matter which table I put first in the query or would the
> >> >> optimizer take care of that?
> >> >> 
> >> >> 
> >> >> Cheers,
> >> >> Niels
> ------------------------------------------------------------------------
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20080130/c06b137d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: benjubb.vcf
Type: text/x-vcard
Size: 255 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20080130/c06b137d/attachment.vcf>