[Qgis-developer] [Qgis-user] Caching remote postgis layers?

Tue Aug 6 02:33:50 PDT 2013

Hi all,

I am *guessing* that Manifold is using the .NET Dataset type to cache data,
and that some kind of db manager takes care of the sync.

SQLAlchemy could be used in a python plugin for the same purpose.

Best,
Angelos

On Tue, Aug 6, 2013 at 9:46 AM, Marco Hugentobler <
marco.hugentobler at sourcepole.ch> wrote:

>  Hi Willem
>
> Another thing to explore is the offline editing plugin. It caches postgis
> layers to a local spatialite database and can even be used to synchronise
> edits between local spatialite db and remote postgis db.
>
>
> >Sourcepole have developed (for NIWA, New Zealand) a plugin that supports
> local caching of WMS & WFS data sources. This could probably be adapted to
> support other input >formats. Marco Hugentobler did the coding for this -
> he can perhaps comment?
>
> The plugin saves WMS as local rasters and WFS as local vectors. This is
> similar to save as local vector / raster file by hand. However the plugin
> is more convenient because it manages the local storage and the reference
> to the remote uri.  It has some other nice features like synchronizing /
> offlining in a graphical way. Btw., we hope to include this plugin (a C++
> one) into the main repository after 2.0 feature freeze is over.
>
> Regards,
> Marco
>
> On 06.08.2013 03:20, Brent Wood wrote:
>
>  Hi Willem,
>
>  Why can you not save such a layer or a selection from such a layer, as a
> local shapefile (or other supported format). is this not effectively a
> "local cache"? Delete/overwrite when you desire.
>
>  Where is the performance bottleneck? Postgis, network, QGIS?  Just
> creating a local cache will not necessarily speed things up, if your
> database is on a fast server with a fast network, it may even get slower.
>
>  Sourcepole have developed (for NIWA, New Zealand) a plugin that supports
> local caching of WMS & WFS data sources. This could probably be adapted to
> support other input formats. Marco Hugentobler did the coding for this - he
> can perhaps comment?
>
>  As Nathan says, caching a local snapshot of a dynamic, multi-user
> database will result in something that gets out of sybc with the source
> pretty quickly. Caching a local copy of a largely static database is
> relatively safe.
>
>  Cheers
>
>  Brent
>   ------------------------------
>  *From:* Nathan Woodrow <madmanwoo at gmail.com> <madmanwoo at gmail.com>
> *To:* Willem Buitendyk <willem at pcfish.ca> <willem at pcfish.ca>
> *Cc:* qgis-user <qgis-user at lists.osgeo.org> <qgis-user at lists.osgeo.org>;
> "qgis-developer at lists.osgeo.org" <qgis-developer at lists.osgeo.org>
> <qgis-developer at lists.osgeo.org> <qgis-developer at lists.osgeo.org>
> *Sent:* Tuesday, August 6, 2013 11:07 AM
> *Subject:* Re: [Qgis-user] Caching remote postgis layers?
>
>  Hey Willem,
>
>  I'm not aware of a feature like that.  The best place to ask this
> question to see if anyone is working, or planning to work, on it is the
> developer list.  I have copied it in so it show up there too.
>
>  Caching could be done by storing the returned results in a memory
> spatialite database. The tricky thing is knowing when to get any new
> results from the postgis database when things have changed on the database.
>  How does Manifold handling viewing an area, updating something using SQL
> on the database (not though Manifold) and then looking at that area again?
>
>  - Nathan
>
>
> On Tue, Aug 6, 2013 at 2:44 AM, Willem Buitendyk <willem at pcfish.ca> wrote:
>
> I've noticed looking at the roadmap that caching remote postgis data is
> not on the list.  I love qgis but a lot of my work involves connecting to a
> remote postgis database.  The speed of navigating a large dataset is, at
> the moment, rather unbearable.  I also use Manifold GIS and the difference
> in speed is shocking, presumably because Manifold is caching the dataset.
>  This seems like it should be a relatively easy thing to implement.  Is
> there interest on this board for this feature?  I was thinking at taking a
> stab at contributing to the source code but am wondering about methods to
> achieve data caching most efficiently.  Look forward to others thoughts?
>  Or maybe I missing something that is already in place?
>
> Willem
>
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-user
>
>
>
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-user
>
>
>
>
> _______________________________________________
> Qgis-developer mailing listQgis-developer at lists.osgeo.orghttp://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> --
> Dr. Marco Hugentobler
> Sourcepole -  Linux & Open Source Solutions
> Weberstrasse 5, CH-8004 Zürich, Switzerlandmarco.hugentobler at sourcepole.ch http://www.sourcepole.ch
> Technical Advisor QGIS Project Steering Committee
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>

-- 
Angelos Tzotsos
Remote Sensing Laboratory
National Technical University of Athens
http://users.ntua.gr/tzotsos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20130806/7d4bc78e/attachment-0001.html>