[Tiling] Steps towards a better filemanagement

[Mike]

I too am very interested in this, as managing even a small number of tiles
really becomes burdensome. A transfer of even a small number of tiles is
significantly slower than a single file.

Having a sqlite file is nice - especially for portable applications - but is
still nice for server and desktop applications.

On Mon, Apr 11, 2011 at 6:51 AM, Klokan Petr Pridal <klokan at klokan.cz>wrote:

> Hi Oliver,
>
> I am really looking forward to read the post on your mentioned
> subject: "design of a database schema for a new tile store format that
> supports multiple caches per database and also caches split into
> multiple databases".
>
> There are indeed use-cases where this is exactly what is technically
> required. We use a similar (simplified) approach in-house and I would
> be very glad to standardize this and share our experience with other
> too.
>
> Please post your ideas and draft into this forum - I am sure it will
> be attractive for the readers and we can then discuss this subject
> further.
>
> Best regards,
>
> Klokan Petr Pridal
>
> (the author of GDAL2Tiles and MapTiler)
>
> On Mon, Apr 11, 2011 at 2:29 PM, Oliver Tonnhofer <olt at omniscale.de>
> wrote:
> > Hi Andreas,
> >
> > On 11.04.2011, at 13:38, Andreas Trawoeger wrote:
> >> I'm not really sure if switching to SQLite is going to make
> tile-cache-management a lot easier in the long run. SQLite would be perfect
> for small and somewhat static tilestores or offline caches for mobile
> applications.
> >> But once they get bigger and have constant read & write access databases
> get notorously difficult to backup.
> >
> > Thanks for raising some concerns. It's too easy to neglect the cons when
> discussing these things.
> >
> >> Having thousand of files can be hard to handle. But there are myriad of
> tools like e.g. rsync that makes handling them a lot easier.
> >
> > Copying thousands of small files is much slower then a few big files. I
> have a virtual machine where it takes 2.5 minutes to do a `du -sk
> cache_data` on 300.000 tiles. And 300.000 tiles is a small number. These are
> the issues when we are talking about manageability.
> >
> > For managing SQLite files, there is a backup API that allows copying
> databases without locking issues. http://www.sqlite.org/backup.html
> >
> > My database design provides support splitting a cache into multiple
> databases (e.g. one DB per resolution, or even multiple DBs for large tile
> sets (level > 16)). So for large caches you would have multiple files.
> >
> > Regards,
> > Oliver
> >
> > --
> > Oliver Tonnhofer    | Omniscale GmbH & Co KG    | http://omniscale.de
> > http://mapproxy.org | https://bitbucket.org/olt | @oltonn
> >
> >
> >
> >
> > _______________________________________________
> > Tiling mailing list
> > Tiling at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/tiling
> >
>
>
> --
> Klokan Technologies GmbH
> http://blog.klokantech.com/
> _______________________________________________
> Tiling mailing list
> Tiling at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/tiling
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/tiling/attachments/20110411/ead7bb12/attachment-0001.html