[SAC] Run away process on Projects VM

[Alex Mandel]

On 03/12/2013 03:56 AM, Markus Neteler wrote:
> Hi again,
>
> there is also a crschmidt Python process causing potential troubles:
>
> dmesg
> Mar 11 19:04:30 projects kernel: [7499374.049855] python[20053]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 11 19:14:36 projects kernel: [7499979.356895] python[8413]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 11 20:06:12 projects kernel: [7503075.591039] python[29533]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 11 20:52:28 projects kernel: [7505851.389659] python[11701]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 11 22:24:25 projects kernel: [7511368.687807] TCP: Possible SYN
> flooding on port 80. Sending cookies.  Check SNMP counters.
> Mar 12 02:24:43 projects kernel: [7525786.958763] python[22365]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 12 02:33:05 projects kernel: [7526288.270835] python[21809]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 12 02:42:09 projects kernel: [7526832.527160] python[2854]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> Mar 12 02:44:22 projects kernel: [7526965.583365] python[12697]:
> segfault at 0 ip 00007f1ce8040f32 sp 00007fff44f89908 error 4 in
> libc-2.11.3.so[7f1ce7fc5000+159000]
> ^C
>
>
> projects:/home/neteler# ps -aef | grep python
> root      1861     1  0  2012 ?        00:23:22 /usr/bin/python
> /usr/bin/gunicorn_django --pid /var/run/gunicorn/sr.gunicorn.pid
> --name sr.gunicorn --user crschmidt --group metacrs --daemon
> --log-file /var/log/gunicorn/sr.gunicorn.log --bind=0.0.0.0:8092
> --workers=4 --worker-class=egg:gunicorn#sync --timeout=30
> 10011     7546  1861  0 02:42 ?        00:00:13 /usr/bin/python
> /usr/bin/gunicorn_django --pid /var/run/gunicorn/sr.gunicorn.pid
> --name sr.gunicorn --user crschmidt --group metacrs --daemon
> --log-file /var/log/gunicorn/sr.gunicorn.log --bind=0.0.0.0:8092
> --workers=4 --worker-class=egg:gunicorn#sync --timeout=30
> 10011     7848  1861  0 02:44 ?        00:00:12 /usr/bin/python
> /usr/bin/gunicorn_django --pid /var/run/gunicorn/sr.gunicorn.pid
> --name sr.gunicorn --user crschmidt --group metacrs --daemon
> --log-file /var/log/gunicorn/sr.gunicorn.log --bind=0.0.0.0:8092
> --workers=4 --worker-class=egg:gunicorn#sync --timeout=30
> 10011     9355  1861  0 02:24 ?        00:00:16 /usr/bin/python
> /usr/bin/gunicorn_django --pid /var/run/gunicorn/sr.gunicorn.pid
> --name sr.gunicorn --user crschmidt --group metacrs --daemon
> --log-file /var/log/gunicorn/sr.gunicorn.log --bind=0.0.0.0:8092
> --workers=4 --worker-class=egg:gunicorn#sync --timeout=30
> root     13060     1  0 Mar10 ?        00:03:00 python
> /usr/sbin/denyhosts --daemon --purge --config=/etc/denyhosts.conf
> --config=/etc/denyhosts.conf
> www-data 20596  6443  0 03:53 ?        00:00:00 python
> /osgeo/openlayers/sites/openlayers.org/dev/examples/proxy.cgi
> www-data 20599  6377  0 03:53 ?        00:00:00 python
> /osgeo/openlayers/sites/openlayers.org/dev/examples/proxy.cgi
> www-data 20623  6310  0 03:54 ?        00:00:00 python
> /osgeo/openlayers/sites/openlayers.org/dev/examples/proxy.cgi
> root     20657  6127  0 03:54 pts/1    00:00:00 grep python
> 10011    21808  1861  0 00:43 ?        00:00:30 /usr/bin/python
> /usr/bin/gunicorn_django --pid /var/run/gunicorn/sr.gunicorn.pid
> --name sr.gunicorn --user crschmidt --group metacrs --daemon
> --log-file /var/log/gunicorn/sr.gunicorn.log --bind=0.0.0.0:8092
> --workers=4 --worker-class=egg:gunicorn#sync --timeout=30
>
> No idea what to do about it...
>
> Markus
>

That one runs as a service, so
sudo /etc/init.d/gunicorn restart
I just did that, since apache seems to still be using all 200 of it's 
allowed processes.

Though I don't know what the libc error is about, I wonder if someone is 
overusing the the proxy in some way, or overuse of the openlayers hosted 
js (can't recall if we implemented caching for that).

Thanks,
Alex