[Tilecache] tilecache_seed dies at a specific tile - how can
I skip it? [SOLVED]
Adrian Popa
adrian_gh.popa at romtelecom.ro
Mon Nov 30 08:13:49 EST 2009
Hello Christopher,
Yes, it's way beyond my current skills. I had a shock when I found that
indentation really matters in the case of python!
It would be nice if it is incorporated in the next release, thanks.
Regards,
Adrian
Christopher Schmidt wrote:
> On Fri, Nov 27, 2009 at 11:10:34AM +0200, Adrian Popa wrote:
>
>> Hello everyone,
>>
>> I finally got the time to fix this problem.
>>
>> I just added a try/except block in Client.py to catch the
>> urllib2.HTTPError. It seems to work (for me), but I need to do further
>> tests, to make sure the tiles don't get skewed or something.
>>
>> I would kindly ask one of the developers to have a look over the patch,
>> and if it's ok (it's just some error handling - it shouldn't screw
>> things up), it should be integrated in the next release of tilecache.
>>
>
> Not as is, because it's not an option. Some people would prefer that
> errors actually cause things to stop -- or at the very least, that errors
> cause the user to receive some notification.
>
> I realize that adding it as an option may be beyond your python skills,
> since you mentioned you don't have much in the way of experience with the
> language. I'll try to integrate this patch with an option to turn it on or
> off in the next TileCache release.
>
> The functionality of catching the exception seems fine, however.
>
> -- Chris
>
>
>> Let me know if there is a procedure I should follow when submitting
>> patches/bug-fixes. (also, how to generate the patch).
>>
>> In terms of output, when such an exception is encountered, it displays a
>> message, waits for 5 seconds and goes on (apparently urlib timesout
>> after ~65s):
>> 07 (004624, 005229) = (2582960.0586 5542601.7932 2587852.0284
>> 5547493.7630) [0.0003s : 2826.027/s] 9140/37195
>> 07 (004625, 005229) = (2587852.0284 5542601.7932 2592743.9982
>> 5547493.7630) [0.0003s : 2826.025/s] 9141/37195
>> 07 (004626, 005229) = (2592743.9982 5542601.7932 2597635.9680
>> 5547493.7630) [0.0003s : 2826.017/s] 9142/37195
>> ### Oops - got a HTTP error while generating tile. This tile will be
>> skipped ###
>> 07 (004627, 005229) = (2597635.9680 5542601.7932 2602527.9378
>> 5547493.7630) [70.1914s : 281.353/s] 9143/37195
>> 07 (004628, 005229) = (2602527.9378 5542601.7932 2607419.9076
>> 5547493.7630) [0.0072s : 281.340/s] 9144/37195
>> 07 (004629, 005229) = (2607419.9076 5542601.7932 2612311.8774
>> 5547493.7630) [0.0050s : 281.334/s] 9145/37195
>>
>> In this case, tile 9143 is skipped.
>>
>>
>> Regards,
>> Adrian
>>
>> Adrian Popa wrote:
>>
>>> Thanks for the help,
>>>
>>> Unfortunately I am not a python programmer (perl is my thing), but I
>>> will *try* to have a look in the source code, or try to catch the url
>>> from apache (unfortunately the server is in production and has lots of
>>> requests)...
>>>
>>> Regards,
>>> Adrian
>>>
>>> Christopher Schmidt wrote:
>>>
>>>> On Mon, Oct 26, 2009 at 09:41:22AM +0200, Adrian Popa wrote:
>>>>
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> I'm getting an error while trying to cache offline parts of my map.
>>>>> The caching process runs fine until it hits a particular tile (my
>>>>> guess is it's tile 9143) - when the process dies (after a timeout
>>>>> of 2 minutes).
>>>>>
>>>>> Here are samples of the output:
>>>>>
>>>>> [root at terra utilities]# python
>>>>> /var/www/html/tilecache/tilecache_seed.py rtc_base 1 10 --bbox
>>>>> 2255332.8831578,5336357.2739115,3306188.8761,6183726.4985628
>>>>> ...
>>>>> 07 (004621, 005229) = (2568284.1491 5542601.7932 2573176.1189
>>>>> 5547493.7630) [0.0003s : 2391.549/s] 9137/37195
>>>>> File "/usr/lib/python2.4/urllib2.py", line 480, in http_error_default
>>>>> raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
>>>>> urllib2.HTTPError: HTTP Error 500: Internal Server Error
>>>>>
>>>>> Here is what httpd error_log says:
>>>>> [Mon Oct 26 09:23:13 2009] [warn] [client 127.0.0.1] Timeout
>>>>> waiting for output from CGI script /var/www/cgi-bin/mapserv
>>>>> [Mon Oct 26 09:23:13 2009] [error] [client 127.0.0.1] Premature end
>>>>> of script headers: mapserv
>>>>> [Mon Oct 26 09:25:13 2009] [warn] [client 127.0.0.1] Timeout
>>>>> waiting for output from CGI script /var/www/cgi-bin/mapserv
>>>>>
>>>>>
>>>>> If I try with the --reverse parameter, the same thing happens after
>>>>> a while, but at a different tile.
>>>>>
>>>>> I have two questions:
>>>>> 1. Can I convince tilecache to skip the tiles it can't render,
>>>>> instead of dying?
>>>>>
>>>>>
>>>> Not build in, but you can certainly modify the client to catch errors
>>>> and keep going.
>>>>
>>>>
>>>>
>>>>> 2. What would be the necessary steps I need to take to troubleshoot
>>>>> this problem? (I am generating this cache from a mapserver which is
>>>>> getting its data from shapefiles).
>>>>>
>>>>>
>>>> Look in the apache logs for the mapserver, look at what the URL is,
>>>> and send it to the mapserver. See if it fails in the same way. If so,
>>>> it's a mapserver 'issue' (quotes because it may simply be taking too
>>>> long to do
>>>> this tile, and the actual issue is that you need to bump up your apache
>>>> timeout), and can be treated externally to TileCache.
>>>>
>>>> -- Chris
>>>>
>>>>
>>>>
>>>>> Thanks,
>>>>> Adrian
>>>>>
>>>>> _______________________________________________
>>>>> Tilecache mailing list
>>>>> Tilecache at openlayers.org
>>>>> http://openlayers.org/mailman/listinfo/tilecache
>>>>>
>>>>>
>>>>
>>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Tilecache mailing list
>>> Tilecache at openlayers.org
>>> http://openlayers.org/mailman/listinfo/tilecache
>>>
>>>
>> --
>> --- Adrian Popa
>> NOC Division
>> Network Engineer
>> Divizia Centrul National de Operare Retea
>> Departament Transport IP & Metro
>> Compartiment IP Core & Backbone
>> Phone: +40 21 400 3099
>>
>>
>
>
>> diff -rau TileCache-2.10-py2.4.egg.orig/TileCache/Client.py TileCache-2.10-py2.4.egg/TileCache/Client.py
>> --- TileCache-2.10-py2.4.egg.orig/TileCache/Client.py 2009-10-12 09:29:19.000000000 +0300
>> +++ TileCache-2.10-py2.4.egg/TileCache/Client.py 2009-11-27 10:58:11.000000000 +0200
>> @@ -114,7 +114,11 @@
>> tileStart = time.time()
>> tile = Tile(layer,x,y,z)
>> bounds = tile.bounds()
>> - svc.renderTile(tile,force=force)
>> + try:
>> + svc.renderTile(tile,force=force)
>> + except urllib2.HTTPError:
>> + print "### Oops - got a HTTP error while generating tile. This tile will be skipped ###"
>> + time.sleep(5)
>> total += 1
>> zcount += 1
>> box = "(%.4f %.4f %.4f %.4f)" % bounds
>>
>
>
>
--
---
Adrian Popa
NOC Division
Network Engineer
Divizia Centrul National de Operare Retea
Departament Transport IP & Metro
Compartiment IP Core & Backbone
Phone: +40 21 400 3099
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/tilecache/attachments/20091130/22952efd/attachment.html
More information about the Tilecache
mailing list