[postgis-users] How does PostGIS / PostgreSQL distribute parallel work to cores for GIS type data?

[Lars Aksel Opsahl]

________________________________
>Hei

>

>From: postgis-users <postgis-users-bounces at lists.osgeo.org> on behalf of Marco Boeringa <marco at boeringa.demon.nl>

>Sent: Wednesday, March 4, 2020 2:49 PM

>To: postgis-users at lists.osgeo.org <postgis-users at lists.osgeo.org>

>Subject: Re: [postgis-users] How does PostGIS / PostgreSQL distribute parallel work to cores for GIS type data?

>

>Hi Paul and Darafei,

>

>Thanks for the insightful answers.

>

>I may attempt to do a comparative test using both my multi-threaded

>implementation using the vertex count load balancing between threads,

>and having postgreSQL do its parallel work. But it will require some

>custom development, so I don't know when I will be able to show results.

>


Hi


We at Nibio have been using content based grids (Developed by Nibio) for while where we group things spatially and content based.


The cell's we generate, may have different size but the number of geometries are limited to your selected max.


We here only use bbox and not the number of points, because then we don't need to open each geometries at this first stage.


The first stage where generate cells re single threaded and it has also been working ok with not counting points in our used cases. (There is no problem adding a counter based on number of points)


Here is example from https://github.com/larsop/find-overlap-and-gap where processing time is cut to the half when the number threads are duplicated.


with 20 parallel threads (a more normal sized data set)

time psql sl  -c"CALL find_overlap_gap_run('org_ar.ar250_flate','geo',4258,'sl_lop.ar250_flate_res',20,1000);" 2>> /tmp/analyze.log;

Time: 2128491.119 ms (35:28.491)


with 40 parallel threads (a more normal sized data set)

time psql sl  -c"CALL find_overlap_gap_run('org_ar.ar250_flate','geo',4258,'sl_lop.ar250_flate_res',40,1000);" 2>> /tmp/analyze.log;

Time: 1076647.282 ms (17:56.647)


Lars
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20200304/ec1211e0/attachment.html>