[Qgis-user] FW: timzonefinder vs point in polygon

Tue Feb 9 04:46:18 PST 2021

Below mail was sent to the developers list instead of the user list.

Jeroen

Van: mail at groenebij.nl <mail at groenebij.nl> 
Verzonden: dinsdag 9 februari 2021 12:11
Aan: 'C Hamilton' <adenaculture at gmail.com>; 'qgis-developer' <qgis-developer at lists.osgeo.org>
CC: 'Richard Duivenvoorde' <rdmailings at duif.net>; 'Tim Sutton' <tim at kartoza.com>; 'Nyall Dawson' <nyall.dawson at gmail.com>
Onderwerp: RE: timzonefinder vs point in polygon

Hi Calvin,

I have run a few tests, and might have found the algorithm you need.

First sharing some result here. It might help in understanding what is going on.

On my system (qgis 3.16, windows10, i510600 cpu at 3.3 GHz, 32 GB mem and a nvme SSD), the Vector->Geoprocessing Tools->Intersection took about 3 minutes and 10 seconds.

It also duplicated 32 of the 10000 points, they got 2 different time zones. That might suggest there is something going on with the precision of both the vector layer and the point layer.

Removing those 32 points and running the algorithm again did not change the run time. So no luck there.

Using postgis and sql queries within DB Manager, things were really quick.

Postgresql 11.10 and postgis 2.5.3

SELECT p.id, p.fid, t.tzid

FROM points_10_32 AS p

JOIN timezones AS t

ON st_within (p.geom, t.geom)

This took just 12 seconds.

Using st_intersects in stead of st_within made no difference, also done in 12 seconds.

I must note, when importing the layers into my database, I always check the ‘create spatial index’

So, I also imported the layers into the database without creating the spatial index.

This time the query took 40 seconds. Still, much faster than the Intersection algorithm in Geoprocessing Tools.

A quick Google search (qgis geoprocessing tool intersect vs st_intersects) got me this page:

 <https://gis.stackexchange.com/questions/307511/faster-alternative-to-intersection-in-qgis-to-just-retain-overlapping-features> https://gis.stackexchange.com/questions/307511/faster-alternative-to-intersection-in-qgis-to-just-retain-overlapping-features

And that got me searching for alternatives and I actually found one:

Vector->Datamanagement Tools->join attributes by location

Settings:

input layer = point layer

Join = timezones layer

Geometric predicate = intersects

Join type = one-to-many

This seems to do the same thing as Geoprocessing Tools->Intersection, however, this one just took 2.9 seconds!

There is definitely something different in how these algorithms work and I have no idea what that might be. But the Join Attributes by Location tool might just be what you are looking for.

Also, 3 seconds is even 4 times faster than the sql query, but that is a different issue.

It might actually have something to do with qgis3.12 and above. Running the sql in DB Manager in qgis 3.10 takes 6 seconds compared to 12 seconds (qgis3.16), see this bug report  <https://github.com/qgis/QGIS/issues/38256> https://github.com/qgis/QGIS/issues/38256

Kind regards,

Jeroen

Van: C Hamilton <adenaculture at gmail.com <mailto:adenaculture at gmail.com> > 
Verzonden: vrijdag 5 februari 2021 22:23
Aan: qgis-developer <qgis-developer at lists.osgeo.org <mailto:qgis-developer at lists.osgeo.org> >; Richard Duivenvoorde <rdmailings at duif.net <mailto:rdmailings at duif.net> >; Tim Sutton <tim at kartoza.com <mailto:tim at kartoza.com> >; Nyall Dawson <nyall.dawson at gmail.com <mailto:nyall.dawson at gmail.com> >; Groene Bij <mail at groenebij.nl <mailto:mail at groenebij.nl> >
Onderwerp: timzonefinder vs point in polygon

I need the ability to do a fast time zone lookup in QGIS and found the popular timezonefinder library which I encorported into the Date/Time Tools plugin. See

 <https://github.com/NationalSecurityAgency/qgis-datetimetools-plugin/releases/tag/3.0.1> https://github.com/NationalSecurityAgency/qgis-datetimetools-plugin/releases/tag/3.0.1

Rather than using this, Nyall suggested that I just use the original polygon data set and do a point in polygon lookup. So I have done some testing on a 10,000 random point data set where I was looking up the timezone of each point.

Time results for 10,000 points

Using timezonefinder: 44 seconds

Using the algorithm Vector->Geoprocessing Tools->Intersection: 4 minutes, 4 seconds

I also tried iterating over each of the 10,000 points then testing each of the time zone polygons to see if the point feature was contained in it using this statement. It returned once it found a match.

polyfeature.geometry().contains(point_feature) 

This was definitely not the way to do it and I killed qgis before it finished.

My conclusion is that timezonefinder has been optimized for very fast lookups. I'm sure there is probably a better way for me to work with a point in polygon lookup and if anyone has any further suggestions let me know. I can see games I could play such as first testing the bounding box or setting up lookup tables, but that is what timezonefinder does so for now that is what I will use.

If I have missed some fast point in polygon lookup up in the QGIS API please let me know.

Thanks,

Calvin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20210209/ab76afdd/attachment-0001.html>