<div dir="ltr">About this issue,<div><br></div><div>Diving a bit thru MapProxy code, I installed eventlet and greenlet in order to force MapProxy to use this to control file locks with success.</div><div><br></div><div style>
Althought I had to "force" util/async.py to use evenlet with:</div><div style>eventlet.monkey_patch(socket=True, select=True)<br></div><div style><br></div><div style>I am still running some test in order to find the impact of this change, but now I can get on-the-fly mapnik data, without busting OCCI plugin with concurrent requests.</div>
<div style><br></div><div style>Cheers,</div><div style>Fernando Ribeiro</div><div><br><div class="gmail_extra"><br><div class="gmail_quote">2013/4/5 André Pirard <span dir="ltr"><<a href="mailto:A.Pirard.Papou@gmail.com" target="_blank">A.Pirard.Papou@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>On 2013-04-05 18:31, Fernando Ribeiro
wrote :<br>
</div><div><div class="h5">
<blockquote type="cite">
<div dir="ltr">Hi list,
<div><br>
</div>
<div>I am facing a strange issue related with tile locking,
while running mapproxy under apache httpd with mod_wsgi.</div>
<div><br>
</div>
<div>Apache and cache dirs are owned by the same user and I am
currently serving 'on-the-fly' data from mapnik (2.0.1).</div>
<div><br>
</div>
<div>The stack trace seems familiar with previous
issues here on list:</div>
<div><br>
</div>
<div>...</div>
<div>
<div><i> File
"/usr/lib/python2.6/site-packages/mapproxy/source/mapnik.py",
line 86, in render</i></div>
<div><i> with self.lock():</i></div>
<div><i> File
"/usr/lib/python2.6/site-packages/mapproxy/platform/cpython/lock.py",
line 42, in __enter__</i></div>
<div><i> self.lock()</i></div>
<div><i> File
"/usr/lib/python2.6/site-packages/mapproxy/platform/cpython/lock.py",
line 72, in lock</i></div>
<div><i> raise LockTimeout('another process is still
running with our lock')</i></div>
<div><i>LockTimeout: another process is still running with our
lock</i></div>
<div><br>
</div>
<div>I can check the lock tiles on filesystem, but
they seem empty (0bytes)</div>
<div><br>
</div>
<div>Some config options:</div>
<div><i>client_timeout: 120<br>
</i></div>
<div><i><br>
</i></div>
<div>
<div><i>sources:</i></div>
<div><i> MAPNIK_OSM_OSP:</i></div>
<div><i> type: mapnik</i></div>
<div><i> mapfile: 'mymapfile.xml' </i></div>
<div><i> use_mapnik2: true</i></div>
<div><i> concurrent_requests: 1</i></div>
</div>
</div>
<div><br>
</div>
<div>wsgi config (from mapproxy-util):</div>
<div>
<div><i>from mapproxy.wsgiapp import make_wsgi_app</i></div>
<div><i>application = make_wsgi_app(r'myconfigfile.yaml')</i></div>
</div>
<div><br>
</div>
<div>Any thoughts on this issue?<br>
</div>
<div><br>
</div>
<div>P.S.- I can deliver 'on-the-fly' data with <i>serve-develop</i>
server, but I am facing a concurrency problem with mapnik OCCI
plugin. I have to setup a preforked apache environment to
overcome this problem.</div>
</div>
</blockquote></div></div>
Empty lock files are normal.<br>
If it can help in any way, I sometimes also get flurries of such
messages for a source that likes not responding.<br>
They make nothing worse except the log.<br>
I'm just wondering if aborting with such messages is normal instead
of just timing out silently.<br>
<br>
Cheers, <br>
<br>
<table>
<tbody>
<tr>
<td valign="top">André.</td>
</tr>
</tbody>
</table>
<br>
<br>
<tt>[info] 127.0.0.1 - - [...] "GET /service?... 500 -</tt><tt><br>
</tt><tt>[...] mapproxy.source.wms - WARNING - could not retrieve
WMS map: No response from URL "...": timed out</tt><tt><br>
</tt><tt>[info] 127.0.0.1 - - [...] "..." 200 -</tt><tt><br>
</tt><tt>[...] mapproxy.source.request - INFO - GET ...</tt><tt><br>
</tt><tt>[...] mapproxy.source.wms - WARNING - could not retrieve
WMS map: No response from URL "...": timed out</tt><tt><br>
</tt><tt>[...] mapproxy.wsgiapp - CRITICAL - fatal error in service
for /service ...</tt><tt><br>
</tt><tt><br>
Traceback (most recent call last):</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py",
line 166, in __call__</tt><tt><br>
</tt><tt> resp = self.handlers[handler_name].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38,
in handle</tt><tt><br>
</tt><tt> return self.services[service].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
in handle</tt><tt><br>
</tt><tt> return handler(parsed_req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
in map</tt><tt><br>
</tt><tt> renderer.render(merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
in render</tt><tt><br>
</tt><tt> layer_merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204,
in _single_call</tt><tt><br>
</tt><tt> result = func(*args)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
in _render_layer</tt><tt><br>
</tt><tt> layer_img = layer.get_map(self.query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 404, in get_map</tt><tt><br>
</tt><tt> result = self._image(query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 437, in _image</tt><tt><br>
</tt><tt> tile_collection =
self.tile_manager.load_tile_coords(affected_tile_coords,
with_metadata=query.tiled_only)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126,
in load_tile_coords</tt><tt><br>
</tt><tt> created_tiles = creator.create_tiles(uncached_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227,
in create_tiles</tt><tt><br>
</tt><tt> created_tiles = self._create_meta_tiles(meta_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300,
in _create_meta_tiles</tt><tt><br>
</tt><tt> created_tiles.extend(self._create_meta_tile(meta_tile))</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307,
in _create_meta_tile</tt><tt><br>
</tt><tt> with self.tile_mgr.lock(main_tile):</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 42, in __enter__</tt><tt><br>
</tt><tt> self.lock()</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 72, in lock</tt><div class="im"><tt><br>
</tt><tt> raise LockTimeout('another process is still running
with our lock')</tt><tt><br>
</tt><tt>LockTimeout: another process is still running with our lock</tt><tt><br>
<br>
<br>
</tt></div><tt>Traceback (most recent call last):</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py",
line 166, in __call__</tt><tt><br>
</tt><tt> resp = self.handlers[handler_name].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38,
in handle</tt><tt><br>
</tt><tt> return self.services[service].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
in handle</tt><tt><br>
</tt><tt> return handler(parsed_req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
in map</tt><tt><br>
</tt><tt> renderer.render(merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
in render</tt><tt><br>
</tt><tt> layer_merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204,
in _single_call</tt><tt><br>
</tt><tt> result = func(*args)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
in _render_layer</tt><tt><br>
</tt><tt> layer_img = layer.get_map(self.query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 404, in get_map</tt><tt><br>
</tt><tt> result = self._image(query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 437, in _image</tt><tt><br>
</tt><tt> tile_collection =
self.tile_manager.load_tile_coords(affected_tile_coords,
with_metadata=query.tiled_only)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126,
in load_tile_coords</tt><tt><br>
</tt><tt> created_tiles = creator.create_tiles(uncached_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227,
in create_tiles</tt><tt><br>
</tt><tt> created_tiles = self._create_meta_tiles(meta_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300,
in _create_meta_tiles</tt><tt><br>
</tt><tt> created_tiles.extend(self._create_meta_tile(meta_tile))</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307,
in _create_meta_tile</tt><tt><br>
</tt><tt> with self.tile_mgr.lock(main_tile):</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 42, in __enter__</tt><tt><br>
</tt><tt> self.lock()</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 72, in lock</tt><div class="im"><tt><br>
</tt><tt> raise LockTimeout('another process is still running
with our lock')</tt><tt><br>
</tt><tt>LockTimeout: another process is still running with our lock</tt><tt><br>
<br>
</tt></div><tt>[info] 127.0.0.1 - - [...] "..." 500 -</tt><tt><br>
</tt><tt>[...] mapproxy.source.request - INFO - GET ...</tt><tt><br>
</tt><tt>[...] mapproxy.source.wms - WARNING - could not retrieve
WMS map: No response from URL "...": timed out</tt><tt><br>
</tt><tt>[info] 127.0.0.1 - - [...] "GET ..." 200 -</tt><tt><br>
</tt><tt>[...] mapproxy.wsgiapp - CRITICAL - fatal error in service
for /service </tt><tt><br>
</tt><tt><br>
Traceback (most recent call last):</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py",
line 166, in __call__</tt><tt><br>
</tt><tt> resp = self.handlers[handler_name].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38,
in handle</tt><tt><br>
</tt><tt> return self.services[service].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
in handle</tt><tt><br>
</tt><tt> return handler(parsed_req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
in map</tt><tt><br>
</tt><tt> renderer.render(merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
in render</tt><tt><br>
</tt><tt> layer_merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204,
in _single_call</tt><tt><br>
</tt><tt> result = func(*args)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
in _render_layer</tt><tt><br>
</tt><tt> layer_img = layer.get_map(self.query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 404, in get_map</tt><tt><br>
</tt><tt> result = self._image(query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 437, in _image</tt><tt><br>
</tt><tt> tile_collection =
self.tile_manager.load_tile_coords(affected_tile_coords,
with_metadata=query.tiled_only)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126,
in load_tile_coords</tt><tt><br>
</tt><tt> created_tiles = creator.create_tiles(uncached_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227,
in create_tiles</tt><tt><br>
</tt><tt> created_tiles = self._create_meta_tiles(meta_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300,
in _create_meta_tiles</tt><tt><br>
</tt><tt> created_tiles.extend(self._create_meta_tile(meta_tile))</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307,
in _create_meta_tile</tt><tt><br>
</tt><tt> with self.tile_mgr.lock(main_tile):</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 42, in __enter__</tt><tt><br>
</tt><tt> self.lock()</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 72, in lock</tt><div class="im"><tt><br>
</tt><tt> raise LockTimeout('another process is still running
with our lock')</tt><tt><br>
</tt><tt>LockTimeout: another process is still running with our lock</tt><tt><br>
</tt></div><tt><br>
<br>
Traceback (most recent call last):</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py",
line 166, in __call__</tt><tt><br>
</tt><tt> resp = self.handlers[handler_name].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38,
in handle</tt><tt><br>
</tt><tt> return self.services[service].handle(req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
in handle</tt><tt><br>
</tt><tt> return handler(parsed_req)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
in map</tt><tt><br>
</tt><tt> renderer.render(merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
in render</tt><tt><br>
</tt><tt> layer_merger)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204,
in _single_call</tt><tt><br>
</tt><tt> result = func(*args)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
in _render_layer</tt><tt><br>
</tt><tt> layer_img = layer.get_map(self.query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 404, in get_map</tt><tt><br>
</tt><tt> result = self._image(query)</tt><tt><br>
</tt><tt> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py",
line 437, in _image</tt><tt><br>
</tt><tt> tile_collection =
self.tile_manager.load_tile_coords(affected_tile_coords,
with_metadata=query.tiled_only)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126,
in load_tile_coords</tt><tt><br>
</tt><tt> created_tiles = creator.create_tiles(uncached_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227,
in create_tiles</tt><tt><br>
</tt><tt> created_tiles = self._create_meta_tiles(meta_tiles)</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300,
in _create_meta_tiles</tt><tt><br>
</tt><tt> created_tiles.extend(self._create_meta_tile(meta_tile))</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307,
in _create_meta_tile</tt><tt><br>
</tt><tt> with self.tile_mgr.lock(main_tile):</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 42, in __enter__</tt><tt><br>
</tt><tt> self.lock()</tt><tt><br>
</tt><tt> File
"/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
line 72, in lock</tt><div class="im"><tt><br>
</tt><tt> raise LockTimeout('another process is still running
with our lock')</tt><tt><br>
</tt><tt>LockTimeout: another process is still running with our lock</tt><br>
<br>
<br>
<br>
<br>
</div></div>
<br>_______________________________________________<br>
MapProxy mailing list<br>
<a href="mailto:MapProxy@lists.osgeo.org">MapProxy@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapproxy" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapproxy</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Fernando Ribeiro</div>
</div></div></div>