[MapProxy] SQLite limits

[Fernando Ribeiro]

Hi Oliver and list,

I misspelled my previous question. I was not stating the actual size limit
of an sqlite database, but the "performance limit" for sqlite dbs,  as I
stated in some sqlite users foruns some concerns on dbs above 2Gb.
I guess we will have to do some performance testing in order to check if
this behaviour is true.

On the other hand, of course we don't want the full 2Tb of data in a single
database... we are thinking on a db "switch" scenario for some given zoom
values and on minor scales by regions, as MapProxy deals great with these
features...but this is only a scenario

Looking forward to hear from the new partition system!

Another question;
Is it safe to install your PIL modification package side-by-side with the
"official" PIL package? Should I uninstall it first?

Regards,


2011/11/7 Oliver Tonnhofer <olt at omniscale.de>

>
> On 04.11.2011, at 19:17, Fernando Ribeiro wrote:
> > Currently my company holds about 2Tb of tiles (increasing) and we are
> evaluating upgrading our current tilecache based service, to mapproxy,
> preferably with mbtiles support.
> > Our current major concern is the actual limit of an sqlite database,
> which is stated around 2Gb. If this is true, we would need a lot of
> partitioning...in other words, managing a lot of partitions, would be very
> annoying.
>
> The file size limit of SQLite is 2TB and not 2 GB, but I'm not suggesting
> to use a single file. SQLite locks the whole database for all write
> operations. Adding a single tile is fast and should lock the file only for
> a few ms, but when you remove larger blocks of tiles, you need to do a
> vacuum and this will block the file for much longer. So, having smaller
> files is still advisable.
>
> Partitioning would be done automatically by MapProxy (DB partitioning, not
> FS), but that is not yet implemented.
>
> Regards,
> Oliver
>
> --
> Oliver Tonnhofer    | Omniscale GmbH & Co KG    | http://omniscale.de
> http://mapproxy.org | https://bitbucket.org/olt | @oltonn
>
>
>
>
>


-- 
Fernando Ribeiro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapproxy/attachments/20111107/ed3bc897/attachment.html