[Tilecache] S3 storage and TileCache

[Bruce Rindahl]

Steven 
I just yesterday set up a system just as you described. It is working just as hoped and is very fast when the tiles are on S3.  The server is an Amazon EC2 and the storage points to my S3 account.  A demo site is at:
http://www.lrcwe-data.com/UDFCD_2008.svg
All the images except Shaded Relief (Under Map Layer Controls - left side check boxes) are coming from the Amazon server.  If the tile has not been created it grabs it from my tilecahce/WMS server and stores it on the S3 account.

You are right about the bandwidth issue - If you just use your local server and the S3 account it will be slower than having a dedicated server.  The nice thing about this setup is the cost to up load the tiles is free since you are doing it through an EC2 setup.  You just pay for the server time (max $72/mo) and the total storage on S3.

I set up a EC2 machine using a Debian (lenny) image.  I added Apache2, mod_python, and TileCache via apt-get.  The S3.py module was missing so I added it from my local machine. (Chris?) Finally I uploaded my tilecache.cfg file to the server (/etc/tilecache.cfg) and configured Apache to run TileCache under mod_python.

If you are using Firefox there are two pretty good add-ons  - S3 organizer and Elasticfox.

It was worth the effort.

Some modifications to TileCache would be helpful.  First TileCache puts everything into one bucket named %userid%-tilecache.  It would really help if a bucket could be created for each layer - like the individual directory is created in a file cache.  This way you could archive or update a layer at a time. I see where the bucket name is created but I haven't been able to modify this yet.  In addition,  it would be great to allow individual layers to use different S3 accounts.  This would be like specifying a different directory or drive for each layer in your file cache.  Maybe the layer could override the defaults.

Let me know if you need more info

Bruce Rindahl






Hi all,

I'm looking into using the Amazon S3 support of TileCache. However at  
the moment TileCache is using S3 as storage only and requests the  
tiles from S3 and then sends them to the client, doubling bandwidth  
usage of the server running TileCache. My reason to use S3 is because  
we run out of bandwidth at our setup and we assume Amazon has  
slightly more available  :) 

I still need tilecache.py because of the sheer amount of layers that  
are available (200 ATM) it is impossible to pretile everything so  
tilecache needs to create tiles if they are not yet available. So  
what I'm looking for is a way to let the client request tiles  
directly from S3 when available and let TileCache create the tile on  
S3 when it is not yet available.

Has anyone tried something like this or has anyone ideas how to do  
such a setup?