[pgrouting-dev] SSL SYSCALL error: EOF detected

[Stephen Woodbridge]

Luis,

All these are good points.

It might also be possible for the client to cause things to leak on the 
server. For example, somethings are created on the server for the client 
and they persist for the duration of the the connection.

Another possibility, is that python might be running all the queries in 
the connection within a transaction, and this allocates memory on the 
server to handle a potential rollback.

You could try closing and opening the connection between requests to see 
if it is related to these.

If you know php, you could try rewriting your script into php (which I'm 
more familiar with) and see if your get the same behavior.

Or you could write all your queries to a text file, then run the text 
file as input to psql and see if the memory grows. If it does not then 
this indicates something weird with the python happening.

./mypython > myqueries.sql
psql mydatabase -f myqueries.sql &> myqueries.log

I'm not saying that the problem is not server side only that the 
valgrind logs do not indicate a problem specifically with pgrouting.

At this point we need to divide problem into smaller pieces  like the 
above suggestion. If we can reproduce it without python, then we can 
eliminate that as the issue. If we can reproduce it with psql and text 
file of sql commands then it indicates something is likely leaking in 
pgrouting and we did not hit the right query in valgrind to reproduce it.

pgRouting does not do much/any? clean up on error conditions so this is 
possibly an error to look at.

-Steve

On 2/27/2015 3:23 AM, Luís de Sousa wrote:
> Hi Steve, my comments go below.
>
> On 26 February 2015 at 16:41, Stephen Woodbridge
> <woodbri at swoodbridge.com> wrote:
>> Hi Luis,
>>
>> I looked at the valgrind log and it is basically the same as a single query
>> and I do not see any leaks coming from pgrouting.
>>
>> I think you should start looking at how you are using python.
>> https://www.google.com/?gws_rd=ssl#newwindow=1&q=psycopg2+memory+leak
>
> I have no memory issues client side, even if the python code is minimal.
>
>> Also, looking at your code, you are computing random start and end node_ids
>> for the query:
>>
>> 1) are you sure you have all nodes from 1 to max+1?
>
> Yes.
>
>> 2) it is possible that if a query errors out that we might leak memory and
>> if this is the case you should attach a simple test case and valgrind log to
>> a bug report.
>
> This is a possibility (e.g. network islands). I will follow this lead.
>
>> Also, in your python you are using fetchall() but you are never freeing the
>> results.
>
> Again, this might not be conventional code, but the memory issue is on
> the server.
>
>> Anyway, at this point I do not see a problem with pgrouting and I'm not
>> seeing this issue on any of my 12.04 systems.
>
> At this stage I am also inclined to consider this an issue external to
> pgRouting. However, the fact that you are able to run the script
> without problems clearly points to the server side.
>
> I will keep investigating this problem. If something further comes up
> relevant to pgRouting I will report back.
>
> Thank you and regards,
>
> Luís
> _______________________________________________
> pgrouting-dev mailing list
> pgrouting-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
>