[Tilecache] Tilecache: Large implementation
crschmidt at metacarta.com
Fri Nov 2 09:30:29 EDT 2007
On Fri, Nov 02, 2007 at 01:08:04PM +0100, Espen Isaksen wrote:
> > When you say 'files', do you mean 'tiles'? I don't know how many actual
> > files I have, but I have about 120Gb of caches on the main labs server.
> > TileCache doesn't scale differently with different numbers of files
> > though, so it seems like perhaps that isn't what you meant.
> I did mean tiles. We are looking at a solution where our storage needs
> will be about 2-3 TB(approximately 150 million tiles). You say that
> TileCache work just as well with TB of data as GB of data? Not
> considering the hardware challenges of course(if anybody have
> suggestions for hardware for taking care of this many tiles, I will of
> course be happy).
There is no difference in TileCache scaling between 1 tile and 10
million tiles, by design. It is possible that your filesystem will make
storing this many files difficult, but TileCache doesn't do anything
One thing to be aware of, for exaple, is that TileCache's default
DiskCache expects that directory traversals will be fast: files are
stored many directories deep in order to ensure that the
links-per-directory doesn't approach the 32,000 limit in place on ext3
file systems. This would mean that storing TileCaches on AFS would be
supremely stupid, since AFS is designed such that directory traversals
But that applies to a single tile as much as it does to 10
billion. TileCache is designed to support up to 30 zoom levels for a
tile set on an ext3 filesystem, and I think it works okay for that --
and if it doens't, you can always modify the DiskCache to fix it.
More information about the Tilecache