[Tiling] Re: inodes

[Gabriel Roldan]

hey thanks both for the answers and insights.
S3 seems cool, and is on my personal roadmap for GWC.
But I'm still looking at/experimenting with alternate storage mechanisms for
the tiles on a regular disk so it's not that hard to manage, and am planning
also to deal with the ocean is blue issue. So far it doesn't look like
computing a SHA1 hash on the blob adds too much overhead, and certainly adds
less than the overhead (GWC) imposes on a couple things that I'm working on
too (like 1-1 FileNotFoundException thrown when reseeding, and optimizing
the computation of the tile location)
But yeah, I'm mixing topics, it's 7:30 am and still working on improvements.

Cheers,
Gabriel
On Fri, Feb 18, 2011 at 6:56 AM, Arne Kepp <arne at tiledmarble.org> wrote:

> On 2/18/11 10:41 AM, Frederic Junod wrote:
>
>> It worked but we realized that the tiles directories where impossible
>> to manage: it took more than 2 days to get the dir size with 'du' and
>> rsync never finish (>  3 days to compute the files to transfer).
>>
> A couple of tips:
>
> Old versions of rsync try to build a file list in memory, which is
> hopeless. If you have rsync 3.0.0 or better (on both sides) it should be
> fine and start transfering almost immediately. But netcat'ing tar is more
> efficient.
>
> Some filsystems do accounting on the directories, so du can be fast. But
> that multiplies the overhead when writing new tiles, since 4-5 directories
> have to be updated. Better to just have tiles (or layers, even) on a
> dedicated partition and use df -h.
>
> -Arne
>
>
> _______________________________________________
> Tiling mailing list
> Tiling at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/tiling
>



-- 
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/tiling/attachments/20110218/1e4d8777/attachment.html