[Tilecache] tilecache_seed dies at a specific tile - how can
I skip it? [SOLVED]
Christopher Schmidt
crschmidt at metacarta.com
Mon Nov 30 07:30:23 EST 2009
On Fri, Nov 27, 2009 at 11:10:34AM +0200, Adrian Popa wrote:
> Hello everyone,
>
> I finally got the time to fix this problem.
>
> I just added a try/except block in Client.py to catch the
> urllib2.HTTPError. It seems to work (for me), but I need to do further
> tests, to make sure the tiles don't get skewed or something.
>
> I would kindly ask one of the developers to have a look over the patch,
> and if it's ok (it's just some error handling - it shouldn't screw
> things up), it should be integrated in the next release of tilecache.
Not as is, because it's not an option. Some people would prefer that
errors actually cause things to stop -- or at the very least, that errors
cause the user to receive some notification.
I realize that adding it as an option may be beyond your python skills,
since you mentioned you don't have much in the way of experience with the
language. I'll try to integrate this patch with an option to turn it on or
off in the next TileCache release.
The functionality of catching the exception seems fine, however.
-- Chris
> Let me know if there is a procedure I should follow when submitting
> patches/bug-fixes. (also, how to generate the patch).
>
> In terms of output, when such an exception is encountered, it displays a
> message, waits for 5 seconds and goes on (apparently urlib timesout
> after ~65s):
> 07 (004624, 005229) = (2582960.0586 5542601.7932 2587852.0284
> 5547493.7630) [0.0003s : 2826.027/s] 9140/37195
> 07 (004625, 005229) = (2587852.0284 5542601.7932 2592743.9982
> 5547493.7630) [0.0003s : 2826.025/s] 9141/37195
> 07 (004626, 005229) = (2592743.9982 5542601.7932 2597635.9680
> 5547493.7630) [0.0003s : 2826.017/s] 9142/37195
> ### Oops - got a HTTP error while generating tile. This tile will be
> skipped ###
> 07 (004627, 005229) = (2597635.9680 5542601.7932 2602527.9378
> 5547493.7630) [70.1914s : 281.353/s] 9143/37195
> 07 (004628, 005229) = (2602527.9378 5542601.7932 2607419.9076
> 5547493.7630) [0.0072s : 281.340/s] 9144/37195
> 07 (004629, 005229) = (2607419.9076 5542601.7932 2612311.8774
> 5547493.7630) [0.0050s : 281.334/s] 9145/37195
>
> In this case, tile 9143 is skipped.
>
>
> Regards,
> Adrian
>
> Adrian Popa wrote:
>> Thanks for the help,
>>
>> Unfortunately I am not a python programmer (perl is my thing), but I
>> will *try* to have a look in the source code, or try to catch the url
>> from apache (unfortunately the server is in production and has lots of
>> requests)...
>>
>> Regards,
>> Adrian
>>
>> Christopher Schmidt wrote:
>>> On Mon, Oct 26, 2009 at 09:41:22AM +0200, Adrian Popa wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I'm getting an error while trying to cache offline parts of my map.
>>>> The caching process runs fine until it hits a particular tile (my
>>>> guess is it's tile 9143) - when the process dies (after a timeout
>>>> of 2 minutes).
>>>>
>>>> Here are samples of the output:
>>>>
>>>> [root at terra utilities]# python
>>>> /var/www/html/tilecache/tilecache_seed.py rtc_base 1 10 --bbox
>>>> 2255332.8831578,5336357.2739115,3306188.8761,6183726.4985628
>>>> ...
>>>> 07 (004621, 005229) = (2568284.1491 5542601.7932 2573176.1189
>>>> 5547493.7630) [0.0003s : 2391.549/s] 9137/37195
>>>> File "/usr/lib/python2.4/urllib2.py", line 480, in http_error_default
>>>> raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
>>>> urllib2.HTTPError: HTTP Error 500: Internal Server Error
>>>>
>>>> Here is what httpd error_log says:
>>>> [Mon Oct 26 09:23:13 2009] [warn] [client 127.0.0.1] Timeout
>>>> waiting for output from CGI script /var/www/cgi-bin/mapserv
>>>> [Mon Oct 26 09:23:13 2009] [error] [client 127.0.0.1] Premature end
>>>> of script headers: mapserv
>>>> [Mon Oct 26 09:25:13 2009] [warn] [client 127.0.0.1] Timeout
>>>> waiting for output from CGI script /var/www/cgi-bin/mapserv
>>>>
>>>>
>>>> If I try with the --reverse parameter, the same thing happens after
>>>> a while, but at a different tile.
>>>>
>>>> I have two questions:
>>>> 1. Can I convince tilecache to skip the tiles it can't render,
>>>> instead of dying?
>>>>
>>>
>>> Not build in, but you can certainly modify the client to catch errors
>>> and keep going.
>>>
>>>
>>>> 2. What would be the necessary steps I need to take to troubleshoot
>>>> this problem? (I am generating this cache from a mapserver which is
>>>> getting its data from shapefiles).
>>>>
>>>
>>> Look in the apache logs for the mapserver, look at what the URL is,
>>> and send it to the mapserver. See if it fails in the same way. If so,
>>> it's a mapserver 'issue' (quotes because it may simply be taking too
>>> long to do
>>> this tile, and the actual issue is that you need to bump up your apache
>>> timeout), and can be treated externally to TileCache.
>>>
>>> -- Chris
>>>
>>>
>>>> Thanks,
>>>> Adrian
>>>>
>>>> _______________________________________________
>>>> Tilecache mailing list
>>>> Tilecache at openlayers.org
>>>> http://openlayers.org/mailman/listinfo/tilecache
>>>>
>>>
>>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Tilecache mailing list
>> Tilecache at openlayers.org
>> http://openlayers.org/mailman/listinfo/tilecache
>>
>
>
> --
> --- Adrian Popa
> NOC Division
> Network Engineer
> Divizia Centrul National de Operare Retea
> Departament Transport IP & Metro
> Compartiment IP Core & Backbone
> Phone: +40 21 400 3099
>
> diff -rau TileCache-2.10-py2.4.egg.orig/TileCache/Client.py TileCache-2.10-py2.4.egg/TileCache/Client.py
> --- TileCache-2.10-py2.4.egg.orig/TileCache/Client.py 2009-10-12 09:29:19.000000000 +0300
> +++ TileCache-2.10-py2.4.egg/TileCache/Client.py 2009-11-27 10:58:11.000000000 +0200
> @@ -114,7 +114,11 @@
> tileStart = time.time()
> tile = Tile(layer,x,y,z)
> bounds = tile.bounds()
> - svc.renderTile(tile,force=force)
> + try:
> + svc.renderTile(tile,force=force)
> + except urllib2.HTTPError:
> + print "### Oops - got a HTTP error while generating tile. This tile will be skipped ###"
> + time.sleep(5)
> total += 1
> zcount += 1
> box = "(%.4f %.4f %.4f %.4f)" % bounds
--
Christopher Schmidt
MetaCarta
More information about the Tilecache
mailing list