[pgrouting-dev] SSL SYSCALL error: EOF detected

[Stephen Woodbridge]

Hi Luis,

This is the easiest way to find a memory leak:

http://blog.cleverelephant.ca/2008/08/valgrinding-postgis.html

You only need to run one query that you think is leaking and this will 
report what, how much and where in the code.

-Steve


On 2/23/2015 7:52 AM, Luís de Sousa wrote:
> Dear Stephen et al.
>
> This seems to be a memory leak in pgr_dijkstra; RAM slowly swells up
> until malloc fails. It happens after some 180 000 paths have been
> calculated, meaning that each call is leaving behind about 10 Kb of
> memory.
>
> A stack trace is attached. I do not have the debug symbols for libc,
> but for postgres it seems to be all there.
>
> I'll keep you posted if I find something more.
>
> Regards,
>
> Luís
>
> On 20 February 2015 at 15:16, Stephen Woodbridge
> <woodbri at swoodbridge.com> wrote:
>> On 2/20/2015 5:58 AM, Luís de Sousa wrote:
>>>
>>> Dear all,
>>>
>>> I am presently implementing the Optimal Meeting Point (OMP) [0]
>>> algorithm with PyWPS. I have a draft brute force algorithm that in
>>> smaller networks computes the OMP correctly by testing all nodes of
>>> the network with pgr_dijkstra (I am not yet using heuristics since
>>> costs are not only distance dependent).
>>>
>>> I am now testing this algorithm in a larger network, of some 50 000
>>> nodes. I previously reported that for about 7%-8% of the nodes
>>> pgr_dijkstra fails, returning an empty tuple [1]. After patching these
>>> exceptions, I am stumping into another issue: somewhere between an
>>> half and two thirds of the network have been processed, Postgres
>>> crashes and I get the error: "SSL SYSCALL error: EOF detected" [2, 3].
>>> This crash seems to happen randomly, with pgr_dijkstra invoked on
>>> different nodes.
>>
>>
>> Hi Luís,
>>
>> The error: "SSL SYSCALL error: EOF detected" is because the postgresql
>> backend has crashed and you connection in python has terminated.
>>
>>> This issue should also be easy to circumvent, but it seems to be
>>> symptom of something a bit more serious with pgrouting.
>>
>>
>> There are a number of things that can crash the database, like running out
>> of memory, or running into a bug in the code.
>>
>> google: gdb postgres stack trace
>>
>> There are a couple of ways to get a stack trace of a crash.
>> 1. is to enable core files
>> 2. is to run the database under gdb
>>
>> HTH,
>>    -Steve
>>
>>> Regards,
>>>
>>> Luís
>>>
>>> [0] https://github.com/pgRouting/pgrouting/issues/289
>>>
>>> [1]
>>> http://pgrouting-users.974093.n3.nabble.com/pgrouting-users-Empty-tuple-from-pgr-dijkstra-tp4025706.html
>>>
>>> [2]
>>> http://stackoverflow.com/questions/20201711/pg-internalerror-ssl-syscall-error-eof-detected
>>>
>>> [3]
>>> http://stackoverflow.com/questions/20217571/psycopg2-interfaceerror-connection-already-closed-pgr-astar
>>> _______________________________________________
>>> pgrouting-dev mailing list
>>> pgrouting-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
>>>
>>
>> _______________________________________________
>> pgrouting-dev mailing list
>> pgrouting-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
>>
>>
>> _______________________________________________
>> pgrouting-dev mailing list
>> pgrouting-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/pgrouting-dev