[MapProxy] LockTimeout issue (mapnik)
Fernando Ribeiro
fernandinand at gmail.com
Fri Apr 5 16:42:26 PDT 2013
About this issue,
Diving a bit thru MapProxy code, I installed eventlet and greenlet in order
to force MapProxy to use this to control file locks with success.
Althought I had to "force" util/async.py to use evenlet with:
eventlet.monkey_patch(socket=True, select=True)
I am still running some test in order to find the impact of this change,
but now I can get on-the-fly mapnik data, without busting OCCI plugin with
concurrent requests.
Cheers,
Fernando Ribeiro
2013/4/5 André Pirard <A.Pirard.Papou at gmail.com>
> On 2013-04-05 18:31, Fernando Ribeiro wrote :
>
> Hi list,
>
> I am facing a strange issue related with tile locking, while running
> mapproxy under apache httpd with mod_wsgi.
>
> Apache and cache dirs are owned by the same user and I am currently
> serving 'on-the-fly' data from mapnik (2.0.1).
>
> The stack trace seems familiar with previous issues here on list:
>
> ...
> * File "/usr/lib/python2.6/site-packages/mapproxy/source/mapnik.py",
> line 86, in render*
> * with self.lock():*
> * File
> "/usr/lib/python2.6/site-packages/mapproxy/platform/cpython/lock.py", line
> 42, in __enter__*
> * self.lock()*
> * File
> "/usr/lib/python2.6/site-packages/mapproxy/platform/cpython/lock.py", line
> 72, in lock*
> * raise LockTimeout('another process is still running with our lock')*
> *LockTimeout: another process is still running with our lock*
>
> I can check the lock tiles on filesystem, but they seem empty (0bytes)
>
> Some config options:
> *client_timeout: 120
> *
> *
> *
> *sources:*
> * MAPNIK_OSM_OSP:*
> * type: mapnik*
> * mapfile: 'mymapfile.xml' *
> * use_mapnik2: true*
> * concurrent_requests: 1*
>
> wsgi config (from mapproxy-util):
> *from mapproxy.wsgiapp import make_wsgi_app*
> *application = make_wsgi_app(r'myconfigfile.yaml')*
>
> Any thoughts on this issue?
>
> P.S.- I can deliver 'on-the-fly' data with *serve-develop* server, but I
> am facing a concurrency problem with mapnik OCCI plugin. I have to setup a
> preforked apache environment to overcome this problem.
>
> Empty lock files are normal.
> If it can help in any way, I sometimes also get flurries of such messages
> for a source that likes not responding.
> They make nothing worse except the log.
> I'm just wondering if aborting with such messages is normal instead of
> just timing out silently.
>
> Cheers,
>
> André.
>
> [info] 127.0.0.1 - - [...] "GET /service?... 500 -
> [...] mapproxy.source.wms - WARNING - could not retrieve WMS map: No
> response from URL "...": timed out
> [info] 127.0.0.1 - - [...] "..." 200 -
> [...] mapproxy.source.request - INFO - GET ...
> [...] mapproxy.source.wms - WARNING - could not retrieve WMS map: No
> response from URL "...": timed out
> [...] mapproxy.wsgiapp - CRITICAL - fatal error in service for /service ...
>
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py", line 166, in
> __call__
> resp = self.handlers[handler_name].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38, in
> handle
> return self.services[service].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
> in handle
> return handler(parsed_req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
> in map
> renderer.render(merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
> in render
> layer_merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204, in
> _single_call
> result = func(*args)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
> in _render_layer
> layer_img = layer.get_map(self.query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 404, in
> get_map
> result = self._image(query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 437, in
> _image
> tile_collection =
> self.tile_manager.load_tile_coords(affected_tile_coords,
> with_metadata=query.tiled_only)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126, in
> load_tile_coords
> created_tiles = creator.create_tiles(uncached_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227, in
> create_tiles
> created_tiles = self._create_meta_tiles(meta_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300, in
> _create_meta_tiles
> created_tiles.extend(self._create_meta_tile(meta_tile))
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307, in
> _create_meta_tile
> with self.tile_mgr.lock(main_tile):
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 42, in __enter__
> self.lock()
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 72, in lock
>
> raise LockTimeout('another process is still running with our lock')
> LockTimeout: another process is still running with our lock
>
>
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py", line 166, in
> __call__
> resp = self.handlers[handler_name].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38, in
> handle
> return self.services[service].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
> in handle
> return handler(parsed_req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
> in map
> renderer.render(merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
> in render
> layer_merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204, in
> _single_call
> result = func(*args)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
> in _render_layer
> layer_img = layer.get_map(self.query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 404, in
> get_map
> result = self._image(query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 437, in
> _image
> tile_collection =
> self.tile_manager.load_tile_coords(affected_tile_coords,
> with_metadata=query.tiled_only)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126, in
> load_tile_coords
> created_tiles = creator.create_tiles(uncached_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227, in
> create_tiles
> created_tiles = self._create_meta_tiles(meta_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300, in
> _create_meta_tiles
> created_tiles.extend(self._create_meta_tile(meta_tile))
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307, in
> _create_meta_tile
> with self.tile_mgr.lock(main_tile):
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 42, in __enter__
> self.lock()
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 72, in lock
>
> raise LockTimeout('another process is still running with our lock')
> LockTimeout: another process is still running with our lock
>
> [info] 127.0.0.1 - - [...] "..." 500 -
> [...] mapproxy.source.request - INFO - GET ...
> [...] mapproxy.source.wms - WARNING - could not retrieve WMS map: No
> response from URL "...": timed out
> [info] 127.0.0.1 - - [...] "GET ..." 200 -
> [...] mapproxy.wsgiapp - CRITICAL - fatal error in service for /service
>
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py", line 166, in
> __call__
> resp = self.handlers[handler_name].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38, in
> handle
> return self.services[service].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
> in handle
> return handler(parsed_req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
> in map
> renderer.render(merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
> in render
> layer_merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204, in
> _single_call
> result = func(*args)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
> in _render_layer
> layer_img = layer.get_map(self.query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 404, in
> get_map
> result = self._image(query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 437, in
> _image
> tile_collection =
> self.tile_manager.load_tile_coords(affected_tile_coords,
> with_metadata=query.tiled_only)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126, in
> load_tile_coords
> created_tiles = creator.create_tiles(uncached_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227, in
> create_tiles
> created_tiles = self._create_meta_tiles(meta_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300, in
> _create_meta_tiles
> created_tiles.extend(self._create_meta_tile(meta_tile))
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307, in
> _create_meta_tile
> with self.tile_mgr.lock(main_tile):
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 42, in __enter__
> self.lock()
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 72, in lock
>
> raise LockTimeout('another process is still running with our lock')
> LockTimeout: another process is still running with our lock
>
>
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py", line 166, in
> __call__
> resp = self.handlers[handler_name].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38, in
> handle
> return self.services[service].handle(req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30,
> in handle
> return handler(parsed_req)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111,
> in map
> renderer.render(merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 471,
> in render
> layer_merger)
> File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204, in
> _single_call
> result = func(*args)
> File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 513,
> in _render_layer
> layer_img = layer.get_map(self.query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 404, in
> get_map
> result = self._image(query)
> File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 437, in
> _image
> tile_collection =
> self.tile_manager.load_tile_coords(affected_tile_coords,
> with_metadata=query.tiled_only)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126, in
> load_tile_coords
> created_tiles = creator.create_tiles(uncached_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227, in
> create_tiles
> created_tiles = self._create_meta_tiles(meta_tiles)
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 300, in
> _create_meta_tiles
> created_tiles.extend(self._create_meta_tile(meta_tile))
> File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 307, in
> _create_meta_tile
> with self.tile_mgr.lock(main_tile):
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 42, in __enter__
> self.lock()
> File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py",
> line 72, in lock
>
> raise LockTimeout('another process is still running with our lock')
> LockTimeout: another process is still running with our lock
>
>
>
>
>
> _______________________________________________
> MapProxy mailing list
> MapProxy at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapproxy
>
>
--
Fernando Ribeiro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapproxy/attachments/20130406/5b9142d2/attachment-0001.html>
More information about the MapProxy
mailing list