[SAC] xblade14-2: Service Temporarily Unavailable (Apache) - gdal, grass, qgis, rso, mapbuilder

[Markus Neteler]

Hi SAC,

I have the problem that this page (which contains a series of screenshot
thumbnails) doesn't load completely - happens frequently to me on
that page:
http://grass.osgeo.org/screenshots/gui.php

Reloading it a few times lead to 503

Service Temporarily Unavailable

The server is temporarily unable to service your request due to
maintenance downtime or capacity problems. Please try again later.
Apache/2.0.54 (Fedora) Server at grass.osgeo.org Port 80

[root at xblade14-2 httpd]# tail grass_error_log
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:19 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/gui.php
[Mon Sep 15 01:24:41 2008] [error] [client 87.5.230.95] Rejecting
client at 87.5.230.95, referer:
http://grass.osgeo.org/screenshots/index.php
[Mon Sep 15 01:25:12 2008] [error] [client 141.223.191.205] Rejecting
client at 141.223.191.205, referer:
http://grass.osgeo.org/wiki/Main_Page
[Mon Sep 15 01:25:12 2008] [error] [client 141.223.191.205] Rejecting
client at 141.223.191.205, referer:
http://grass.osgeo.org/wiki/Main_Page
[root at xblade14-2 httpd]# host 87.5.230.95
95.230.5.87.in-addr.arpa domain name pointer
host95-230-dynamic.5-87-r.retail.telecomitalia.it.

It seems that Apache is severly suffering.
But the CPU(s) aren't doing much:

[neteler at xblade14-2 ~]$ w
 01:24:49 up 47 days,  2:10,  1 user,  load average: 0.33, 0.45, 0.61
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
neteler  pts/0    host95-230-dynam 01:21    1.00s  0.05s  0.02s w

[neteler at xblade14-2 ~]$ top
top - 01:24:51 up 47 days,  2:10,  1 user,  load average: 0.30, 0.45, 0.61
Tasks: 119 total,   1 running, 118 sleeping,   0 stopped,   0 zombie
Cpu(s): 21.2% us,  2.4% sy,  0.3% ni, 68.9% id,  7.0% wa,  0.2% hi,  0.0% si
Mem:   1034320k total,  1006748k used,    27572k free,    29760k buffers
Swap:  2096472k total,   136612k used,  1959860k free,   396052k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  850 neteler   15   0  4752 1520 1128 R  1.9  0.1   0:00.02 top
    1 root      16   0  1744  512  488 S  0.0  0.0   0:03.69 init
    2 root      34  19     0    0    0 S  0.0  0.0   0:02.16 ksoftirqd/0
...

[root at xblade14-2 httpd]# grep 'Rejecting client' grass_error_log | wc -l
4869

Any clue? Apache seems to be messed up:

[root at xblade14-2 httpd]# grep -l 'Rejecting client' *_error_log
gdal_error_log
grass_error_log
mapbuilder_error_log
qgis_error_log
rso_error_log

It affects all of us who are hosted there.

Markus