[Tilecache] Tilecache: Large implementation

[Espen Isaksen]

Ok, thanks a lot for your answers and clarifications!

Espen


On 02/11/2007, Christopher Schmidt <crschmidt at metacarta.com> wrote:
> On Fri, Nov 02, 2007 at 01:08:04PM +0100, Espen Isaksen wrote:
> > > When you say 'files', do you mean 'tiles'? I don't know how many actual
> > > files I have, but I have about 120Gb of caches on the main labs server.
> > > TileCache doesn't scale differently with different numbers of files
> > > though, so it seems like perhaps that isn't what you meant.
> >
> >
> > I did mean tiles. We are looking at a solution where our storage needs
> > will be about 2-3 TB(approximately 150 million tiles). You say that
> > TileCache work just as well with TB of data as GB of data? Not
> > considering the hardware challenges of course(if anybody have
> > suggestions for hardware for taking care of this many tiles, I will of
> > course be happy).
>
> There is no difference in TileCache scaling between 1 tile and 10
> million tiles, by design. It is possible that your filesystem will make
> storing this many files difficult, but TileCache doesn't do anything
> different.
>
> One thing to be aware of, for exaple, is that TileCache's default
> DiskCache expects that directory traversals will be fast: files are
> stored many directories deep in order to ensure that the
> links-per-directory doesn't approach the 32,000 limit in place on ext3
> file systems. This would mean that storing TileCaches on AFS would be
> supremely stupid, since AFS is designed such that directory traversals
> are slow.
>
> But that applies to a single tile as much as it does to 10
> billion. TileCache is designed to support up to 30 zoom levels for a
> tile set on an ext3 filesystem, and I think it works okay for that --
> and if it doens't, you can always modify the DiskCache to fix it.
>
> Regards,
> --
> Christopher Schmidt
> MetaCarta
>