[MapProxy] "Stuck" requests and Apache

[Paul Norman]

I recently deployed mapproxy 1.6.0 with two layers and set up a demo 
page showing those two layers at http://tile.paulnorman.ca/kelowna.html. 
Last night I was having issues with apache becoming nonresponsive, so I 
switched to the prefork MPM as suggested in the mapproxy documentation, 
then put out a mailing list announcement. 

This morning I woke up and checked /server-status on the machine. I see 
there are 91 requests in the "sending reply" stage, all which have been 
for at least 1 hour, and all on the overlay layer. 

Two typical lines are
Srv    PID    Acc     M  CPU     SS    Req  Conn  Child  Slot  Client  VHost
Request
122-0  28898  0/17/17 W  0.32    5948  0    0.0   0.56   0.56  xxx
tile.paulnorman.ca  GET /kelowna_overlay/20/176471/356076.
128-0  29331  0/0/0   W  0.00    5940  0    0.0   0.00   0.00  yyy
tile.paulnorman.ca  GET /kelowna_overlay/18/44117/89020.pn

My config has two layers, an imagery layer and an overlay layer. The 
relevant source definitions are
https://github.com/osm-ca/mapproxy-config-faramir/blob/40d95e16/mapproxy.yam
l#L35-L45.
The imagery layer is handled by mapserver with the source data coming 
from an ECW and appears to be working without problems. The overlay layer 
is rendered by mapnik with the data coming from postgresql 9.3 + postgis
2.1.

Mapproxy is deployed with Apache/2.2.22 (Ubuntu) mod_wsgi/3.3 Python/2.7.3

The machine is currently idle with a load average <0.1, CPU usage <0.5% 
and no measurable IO.

`ps aux' shows 102 processes, all <0.5% CPU, 0.5% memory, assorted start 
times.

The machine has 8GB RAM and an i3 processor with 4 threads and 2 cores.

The only differences I can see between the two layers are
- mapnik vs mapserver
- png8 vs 32-bit PNGA
- concurrent_requests

I'm going to try adjusting concurrent_requests and see if that helps 
matters, but otherwise I'm out of ideas.