Hi Oliver!<br><br><div class="gmail_quote">2011/4/11 Oliver Tonnhofer <span dir="ltr"><<a href="mailto:olt@omniscale.de">olt@omniscale.de</a></span>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div class="im"><br>
On 11.04.2011, at 13:38, Andreas Trawoeger wrote:<br>
> I'm not really sure if switching to SQLite is going to make tile-cache-management a lot easier in the long run. SQLite would be perfect for small and somewhat static tilestores or offline caches for mobile applications.<br>
> But once they get bigger and have constant read & write access databases get notorously difficult to backup.<br>
<br>
</div>Thanks for raising some concerns. It's too easy to neglect the cons when discussing these things.<br>
<div class="im"><br>
> Having thousand of files can be hard to handle. But there are myriad of tools like e.g. rsync that makes handling them a lot easier.<br>
<br>
</div>Copying thousands of small files is much slower then a few big files. I have a virtual machine where it takes 2.5 minutes to do a `du -sk cache_data` on 300.000 tiles. And 300.000 tiles is a small number. These are the issues when we are talking about manageability.<br>
<br>
For managing SQLite files, there is a backup API that allows copying databases without locking issues. <a href="http://www.sqlite.org/backup.html" target="_blank">http://www.sqlite.org/backup.html</a><br>
<br>
My database design provides support splitting a cache into multiple databases (e.g. one DB per resolution, or even multiple DBs for large tile sets (level > 16)). So for large caches you would have multiple files.<br>
</blockquote><div><br>It always depends on where your bottleneck is.<br><br>My 1&1 virtual server for 30 Euro/month currently holds 1.339.303 tiles and doing a `du -sk cache_data` needs 1m 19sek, and it can easily saturate its 100Mbit internet uplink, even when copying small tiles. So having lots of small files never has been a problem to me.<br>
<br>Things definitely change once you run a server with multiple Gigabit Ethernet connections or use a backup solution like e.g. TSM [0] that stores the location of every file into a database.<br><br><br><br>cu andreas<br>
</div></div><br><br>[0] <a href="https://secure.wikimedia.org/wikipedia/en/wiki/IBM_Tivoli_Storage_Manager">https://secure.wikimedia.org/wikipedia/en/wiki/IBM_Tivoli_Storage_Manager</a><br>