Hi Kurt,<div><br></div><div>Thank you for reporting this issue! <br><div><br></div><div>Without being able to answer your questions about the memory allocation, I would like to point you to some new algorithms developed during the Google Summer of Code program this year.</div>
<div><br></div><div>Jay as well as Kishore (our sutedents) have added an "All-Pair-Shortest-Path" (APSP) function, that can calculate a distance matrix. Doing many shortest path queries is inefficient, because for every query it selects the network into the memory again and again. APSP should do this only once, which should also solve the memory issue.</div>
<div><br></div><div>Currently Jay's and Kishore's code resides in different branches of the Github repository. And you would have to take the role as a "beta-tester" ;-) </div><div>Though if you often need to calculate a distance matrix then it might be worth to try. Of course your feedback would be very welcome!</div>
<div><br></div><div>Here a few links if you want to take a look.</div><div><ul><li><a href="https://github.com/pgRouting/pgrouting/network">https://github.com/pgRouting/pgrouting/network</a> <br>look for branch "gsoc-tdsp" and "apsp-johnson" </li>
<li><a href="https://github.com/pgRouting/pgrouting/wiki/APSP">https://github.com/pgRouting/pgrouting/wiki/APSP</a><br>This document should be for APSP in the "gsoc-tdsp" branch. Jay, can you confirm this? Is the document still valid?</li>
</ul>Best regards,</div><div>Daniel </div><div><br><br><div class="gmail_quote">On Wed, Aug 31, 2011 at 4:04 AM, Weninger Kurt <span dir="ltr"><<a href="mailto:kurt.weninger@utanet.at">kurt.weninger@utanet.at</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hi !<br>
<br>
I am calculating the shortest paths for some nodes of a large network with<br>
about 600.000 nodes and edges. Building network topology works fine, also<br>
computing the distance for one node.<br>
The problem I am encountering occurs when I am trying to calculate the<br>
shortest path matrix for several selected nodes (about 1000) at once. I am<br>
calling the function for each node and as I also have reverse costs are<br>
getting the resulting matrix.<br>
Problem is that I get a segfault in librouting_dd.so after ~ 170 nodes in<br>
Windows and ~ 300 nodes in Linux. The backtrace leads me to line 162 of<br>
boost_drivedist.cpp.<br>
When I am doing the calculation in pieces of 100 nodes and logging out<br>
between calculations everything works fine.<br>
<br>
I think the problem is using malloc to reserve memory, as I observed<br>
increasing memory usage of process postgres.<br>
<br>
*path = (path_element_t *) malloc( sizeof(path_element_t) *<br>
(path_vector.size() + 1) );<br>
<br>
It seems to me the memory is never freed.<br>
Can this be solved using palloc ? And how ? Because I tried changing<br>
malloc to palloc, but getting strange errors in other libraries<br>
afterwards.<br>
<br>
Thanks for your help,<br>
<br>
Kurt<br>
<br>
<br>
<br>
_______________________________________________<br>
Pgrouting-users mailing list<br>
<a href="mailto:Pgrouting-users@lists.osgeo.org">Pgrouting-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/pgrouting-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/pgrouting-users</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><span style="font-family:arial, sans-serif;font-size:13px;border-collapse:collapse">Georepublic UG & Georepublic Japan<br>eMail: <a href="mailto:daniel.kastl@georepublic.de" style="color:rgb(66, 99, 171)" target="_blank">daniel.kastl@georepublic.de</a><br>
Web: <a href="http://georepublic.de/" style="color:rgb(66, 99, 171)" target="_blank">http://georepublic.de</a></span><br>
</div></div>