[gdal-dev] Adding Sync to CouchDB/Cloudant Driver

[Norman Barker]

Hi Even,

the aim is to sync data between masters as per -
http://dataprotocols.org/couchdb-replication/ An example would be using
GDAL in QGIS and going offline to edit data and then coming back online,
the user would need to synchronize the dataset. As replication is a core
part of CouchDB I would like to add it to the CouchDB OGR Driver.

Synchronizing at the database opening is a great idea, that makes sense.

LevelDB would be a new GDAL dependency for the CouchDB driver, the GeoJSON
will be persisted in leveldb so that subsequent connections to a couchdb
database would only retrieve any changes to an upstream CouchDB server.
LevelDB is under the BSD license. LevelDB is used in PouchDB so it is
relatively easy to follow that code. I would aim to abstract leveldb behind
a OGRCouchDBCache class so that this storage engine could be changed.

Norman



On Wed, Mar 4, 2015 at 1:43 AM, Even Rouault <even.rouault at spatialys.com>
wrote:

> Norman,
>
> I'm not familiar with what this "sync" is about, so please bear with the
> naive
> questions.
> Is it to synchronize all masters between them ?
> Why GetFeature() and/or not at the first GetNextFeature(), or at database
> opening ?
> What is the persistence you mention, and what is the connexion with the
> sync
> you mention ? Is it done at client (OGR) side ? Would leveldb be a new GDAL
> dependency ?
>
> Even
>
> > Hi,
> >
> > I am looking to add sync to the GDAL CouchDB driver. I would rather not
> add
> > additional API functions to do this but to wrap it up in the
> > usual GetFeature call. The first call to GetFeature would initiate a
> sync,
> > but thoughts on this are appreciated.
> >
> > Sync is a useful function that is a key differentiator with CouchDB which
> > is a multi-master database.
> >
> > There are a couple of ways of doing the sync, I could force a whole
> > database sync (the CouchDB model) or we could do a filtered sync based on
> > the bbox of the current local dataset. I would do the former in the
> initial
> > implementation.
> >
> > As for the persistence, I would like to use leveldb as a local store
> though
> > I could use sqlite. The two schemas are as follows;
> >
> >
> https://github.com/pouchdb/pouchdb/blob/master/lib/adapters/idb/idb.js#L56
> >
> https://github.com/couchbaselabs/TouchDB-iOS/wiki/Object-Design-And-Schema
> >
> > I will be create a PR on this shortly, but I am interested in any early
> > feedback.
> >
> > Norman
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20150304/aeacf8da/attachment-0001.html>