[postgis-users] Parallelisation provides powerful postgis performance perks (script + ppt slides) [x-posted: pgsql-performance]

[Kuien Liu]

Great, see you at FOSS4G

Cheers,
Kuien Liu

On Thu, Jul 23, 2015 at 10:24 PM, David Haynes <haynesd2 at gmail.com> wrote:

> Hello,
>
> I am a researcher at the University of Minnesota, US. who is working on a
> project that uses PostGIS as our platform. I have been researching a number
> of PostgreSQL platforms that claim to support parallelizing PostGIS or
> geographic analysis. I am interested to learn more about the wrapper you
> have written. Have you looked into the other platforms such at pg_shard
> (CitusDB), or postgres-xl? We have not found them effective for actually
> distributing a spatial query. However, I will test your code out in our use
> case.
>
> When I briefly look at the text you have written in the "Quick Example" It
> seems that you are distributing your query by an ID field. I am wondering
> how your method would apply to raster datasets? Distributing geographic
> data by an ID can get you into problems because of the dependency for
> certain analytical functions.
>
> This sounds great, hope to hear back from you soon.
>
> On Thu, Jul 23, 2015 at 8:54 AM, Graeme B. Bell <graeme.bell at nibio.no>
> wrote:
>
>> Hi all,
>>
>> Do you run map intersections between national scale geometry maps in
>> postgis? (or other long-running GIS operations?)
>> Do you hate that feeling of waiting all day/week for the query to
>> complete?
>> Well, here's a solution for you.
>>
>> 1. For those that don't like par_psql (http://github.com/gbb/par_psql),
>> this alternative approach uses the Gnu Parallel command to organise
>> parallelism for queries that take days to run usually. We saw up to 20x
>> performance improvements here, a day's work in one hour. May give you a few
>> ideas about how to parallelise your own code with Gnu Parallel.
>>
>> https://github.com/gbb/fast_map_intersection
>>
>>
>> 2. Also, I gave a talk at FOSS4G Como about these tools (and a few
>> others), and how to get better performance from your PostGIS database with
>> parallelisation.
>> This may be helpful to people who are new to parallelisation / multi-core
>> work with postgres for GIS work.
>>
>> http://graemebell.net/foss4gcomo.pdf
>>
>> Enjoy,
>>
>> Graeme Bell.
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at lists.osgeo.org
>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20150803/7f5f3582/attachment.html>