[Tiling] Steps towards a better filemanagement

[Klokan Petr Pridal]

Hi Oliver,

I am really looking forward to read the post on your mentioned
subject: "design of a database schema for a new tile store format that
supports multiple caches per database and also caches split into
multiple databases".

There are indeed use-cases where this is exactly what is technically
required. We use a similar (simplified) approach in-house and I would
be very glad to standardize this and share our experience with other
too.

Please post your ideas and draft into this forum - I am sure it will
be attractive for the readers and we can then discuss this subject
further.

Best regards,

Klokan Petr Pridal

(the author of GDAL2Tiles and MapTiler)

On Mon, Apr 11, 2011 at 2:29 PM, Oliver Tonnhofer <olt at omniscale.de> wrote:
> Hi Andreas,
>
> On 11.04.2011, at 13:38, Andreas Trawoeger wrote:
>> I'm not really sure if switching to SQLite is going to make tile-cache-management a lot easier in the long run. SQLite would be perfect for small and somewhat static tilestores or offline caches for mobile applications.
>> But once they get bigger and have constant read & write access databases get notorously difficult to backup.
>
> Thanks for raising some concerns. It's too easy to neglect the cons when discussing these things.
>
>> Having thousand of files can be hard to handle. But there are myriad of tools like e.g. rsync that makes handling them a lot easier.
>
> Copying thousands of small files is much slower then a few big files. I have a virtual machine where it takes 2.5 minutes to do a `du -sk cache_data` on 300.000 tiles. And 300.000 tiles is a small number. These are the issues when we are talking about manageability.
>
> For managing SQLite files, there is a backup API that allows copying databases without locking issues. http://www.sqlite.org/backup.html
>
> My database design provides support splitting a cache into multiple databases (e.g. one DB per resolution, or even multiple DBs for large tile sets (level > 16)). So for large caches you would have multiple files.
>
> Regards,
> Oliver
>
> --
> Oliver Tonnhofer    | Omniscale GmbH & Co KG    | http://omniscale.de
> http://mapproxy.org | https://bitbucket.org/olt | @oltonn
>
>
>
>
> _______________________________________________
> Tiling mailing list
> Tiling at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/tiling
>


-- 
Klokan Technologies GmbH
http://blog.klokantech.com/