[pgrouting-dev] out of memory with kdijkstra

[Stephen Woodbridge]

On 12/22/2014 8:06 PM, yobiSource wrote:
> Hi,
>
> is there any progress in memory management? [1]
>
> I create 5000 times a distance matrix (pgr_kdijkstra) with ~300 points
> and run oom. (>20 GB)
>
> I avoid that by just split the data and open multiple connections (one
> after another) but it is a bit complicated.
>
>
> [1]
> https://github.com/pgRouting/pgrouting/issues?q=is%3Aissue+is%3Aopen+memory
>

Hi yobi,

I can appreciate how frustrating issues are like this. They are for the 
developers also. Maybe you should open a ticket for this issue and 
decribe what you are doing in some detail. Ideally getting a small 
database dump and a query that reproduces problem would be very helpful.

While there are other commands that have memory issues, they are most 
likely not related to your OOM issue. Information you should post in the 
ticket includes:

OS: Windows, Linux

Hardware: 32/64 bit, Memory, amount of swapfile space

How many shared_buffers in your postgresql.conf file?

How many edges in your graph?
    select count(*) from edge_table;

edge and node ids:
    select min(source) as smin, max(source) as smaxm, min(target) as 
tmin, max(target) as tmax, min(gid) as gmin, max(gid) as gmax
     from edge_table;

What is your query that is running out on memory?

Describe the steps to reproduce the problem.

Then you can refer to the ticket number when asking about the issue and 
all the data is in one place.

As far as progress, we have been tied up on other projects because we 
get very little funding for pgrouting, but with some more detailed 
information we might be able to help you work around the issue.

Also you might want to look at the tools in github branch develop in 
src/common/sql/pgrouting_conversion_tools.sql which has a lot of tools 
for building distance matrices.

Also src/kdijkstra/src/k_targets_sp.c which has some new stuff that is 
not in the tagged 2.0.0 release.

-Steve