[mapguide-internals] MapGuide RFC 112 - sqlite based tile cache

[Trevor Wekel]

I agree with Traian.  There are alternative solutions for replication and backup.

If we are considering replication and backup for the tile sets, we should also consider replication for the XML definitions (layer,feature,map) used to generate those tiles.  In other words, I would like to consider tile replication and repository replication together.

The replication and backup functionality in MapGuide is certainly lacking.  MGP files do not propagate user/group/role information.  The only "easy" way to back up or replicate an entire server is to stop the MapGuide Server and copy all the files around.  I doubt that Rsync or robocopy could replicate live BerkeleyDB files.

I also took a quick look at SQLite replication.  Google didn't turn up anything that was LGPL and actively maintained.  SQLite does have an internal hook that we could use to replicate stuff stored in SQLite http://www.sqlite.org/capi3ref.html#sqlite3_update_hook.  We could roll our own.

Since we allow access to external data sources (SHP files, ECW files, etc), replication of file based data from server to server would have to be considered as part of the solution.  And replication to a UNC path would be an easy way to implement backup.

Master/Slave replication based on files and SQLite could be implemented in phases.  Here's a very rough outline:

Phase 1 - Tile and external data replication
- Reintroduce master/slave concept for MapGuide Server
- Implement server to server TCP/IP communication logic to transfer files
- Implement local "file copy (UNC backup)" logic
 
Phase 2 - Switch to SQLite for tiles
- Add SQLite to the MapGuide Server.  Recode MgTileService to populate the database
- Implement a SQLite update hook
- Implement server to server TCP/IP communication logic for propagating SQLite updates

Phase 3 - Full repository replication
- Rip out BerkeleyDB and replace it with SQLite
- Use existing mechanism from Phase 2 to implement full replication of repository

Regards, 
Trevor

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: May 12, 2011 11:59 AM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] MapGuide RFC 112 - sqlite based tile cache


Hi Tom,

It would depend on what exactly the problem is -- sure if one is using Windows Explorer drag and drop to backup the files it would be faster to have one file. But if one is using rsync or robocopy (or similar), it still makes sense to use files, since those programs know how to copy only the changed files (or even changed parts of files).

Traian