[pgrouting-users] pgRouting and multi-core

[Yaron Lev]

Steve, thank you very much for your help!

your guys are doing fantastic work, and i really appreciate your help!


On Tue, Jul 23, 2013 at 5:01 PM, Stephen Woodbridge <woodbri at swoodbridge.com
> wrote:

> On 7/23/2013 9:43 AM, Yaron Lev wrote:
>
>> thanks a lot for your in-depth explanation, it helped me a lot and indeed
>> i
>> was referring to the c/c++ part.
>>
>> the basic question i am  trying to understand, how beneficial it will be
>> to
>> increase the DB instance size (cpu wise) (i am using AWS).
>>
>> just to be clear, and re-iterating, each request has two basic parts:
>>
>> 1. grab data, and build the graph on the db - this is done on the
>> PostgreSQL db, and therefor will benefit from multi-core environment.
>>
>
> correct
>
>
>  2. solving and returning the answer  -
>> here i am a little lost. PostgreSQL allow integration of C, i would expect
>> then, that these extensions(pgRouting code) will also benefit from
>> multi-core environment.
>> (i.e each request will use a different thread if needed, and on different
>> core if needed)
>>
>
> yes, each request runs in a separate db instance or thread.
>
>
>  so in short, will part 2, also benefit from increasing the instance size(i
>> understand it consumes roughly 30% of resources needed to complete the
>> request)?
>>
>
> The C/C++ code *mostly* does its own memory allocation and does not rely
> on the memory pooling that postgresql provides. some parts of the progress
> to use this but the graphs and solution memory is allocated outside the
> postgresql memory pooling.
>
> In general if you are not running a lot of other processes on the server
> you can allocate more memory to postgresql, how much memory the graphs use
> depends on the typical size of a graph. I have never tried to figure this
> out. I would go with the standard memory sizing suggestions you can find
> for postgresql tuning. If you need a number to start with, on an 8GB memory
> system you might try 4GB in shared buffers and watch it under load with top
> to see how memory is being used and how much swap is getting used, how
> large the processes are getting, etc. and then make adjustments from there.
>
> Also you can optimize your data storage by clustering on the edge table
> spatial index. This will minimize IO when selecting edges for a graph
> because edges that are spatially close together will get clustered on
> pages, and once you need one edge on a page it is likely you will need
> multiple edges on that page and page caching is done in the shared buffers.
>
> -steve
>
>  again, many thanks for your help.
>>
>>
>>
>>
>> On Tue, Jul 23, 2013 at 3:39 PM, Stephen Woodbridge <
>> woodbri at swoodbridge.com
>>
>>> wrote:
>>>
>>
>>  On 7/23/2013 7:55 AM, Yaron Lev wrote:
>>>
>>>  Hi,
>>>>
>>>> i was looking for some information about weather or not pgRouting is
>>>> utilizing more than one core if available.
>>>>
>>>> from the limited testing i made, and from googling (i found very little
>>>> refrences), i think pgRouting does not support multi-core processing as
>>>> of
>>>> now.
>>>>
>>>> i will appreciate if anyone can confirm it, or give any information
>>>> about
>>>> it.
>>>>
>>>>
>>> PgRouting is built on top of Postgresql so to the extent that postgresql
>>> can use multiple cores to process multiple requests we do. I you are
>>> asking
>>> if the routing algorithm itself is run in threads or using something like
>>> parallel graph algorithms? the answer is no.
>>>
>>> Also you should be aware the solving the graph is less then 50% of the
>>> cost of answering a query because we have to query the database for the
>>> edges, build the graph, solve the graph and return the results back to
>>> the
>>> client. selecting the edges and building the graph likely take 2/3rds of
>>> the processing time for a given query. The edge selection is probably
>>> done
>>> in a separate core/thread/process by the database from that that the rest
>>> of the processing is taking place in.
>>>
>>> -Steve
>>>
>>>
>>> ______________________________****_________________
>>> Pgrouting-users mailing list
>>> Pgrouting-users at lists.osgeo.****org <Pgrouting-users at lists.osgeo.**org<Pgrouting-users at lists.osgeo.org>
>>> >
>>> http://lists.osgeo.org/****mailman/listinfo/pgrouting-****users<http://lists.osgeo.org/**mailman/listinfo/pgrouting-**users>
>>> <http://lists.osgeo.org/**mailman/listinfo/pgrouting-**users<http://lists.osgeo.org/mailman/listinfo/pgrouting-users>
>>> >
>>>
>>>
>>
>>
>> ______________________________**_________________
>> Pgrouting-users mailing list
>> Pgrouting-users at lists.osgeo.**org <Pgrouting-users at lists.osgeo.org>
>> http://lists.osgeo.org/**mailman/listinfo/pgrouting-**users<http://lists.osgeo.org/mailman/listinfo/pgrouting-users>
>>
>>
> ______________________________**_________________
> Pgrouting-users mailing list
> Pgrouting-users at lists.osgeo.**org <Pgrouting-users at lists.osgeo.org>
> http://lists.osgeo.org/**mailman/listinfo/pgrouting-**users<http://lists.osgeo.org/mailman/listinfo/pgrouting-users>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pgrouting-users/attachments/20130723/b36352e0/attachment-0001.html>