<div dir="ltr"><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">SQLAlchemy is a ORM it's not going to help with any kind of caching.</span><br><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px"><br>
</span></div><div><font color="#000000" face="arial, sans-serif">There are plans for creating a in memory feature cache of sorts so that we don't have to fetch each feature everytime however there is a bit of work involved in making sure we update the view when the data source changes.</font></div>
<div><font color="#000000" face="arial, sans-serif"><br></font></div><div><font color="#000000" face="arial, sans-serif">- Nathan</font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Aug 6, 2013 at 7:33 PM, Angelos Tzotsos <span dir="ltr"><<a href="mailto:gcpp.kalxas@gmail.com" target="_blank">gcpp.kalxas@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi all,<div><br></div><div>I am <b>guessing</b> that Manifold is using the .NET Dataset type to cache data, and that some kind of db manager takes care of the sync.</div>
<div><br></div><div>SQLAlchemy could be used in a python plugin for the same purpose.</div>
<div><br></div><div>Best,</div><div>Angelos</div><div> </div></div><div class="gmail_extra"><div><div class="h5"><br><br><div class="gmail_quote">On Tue, Aug 6, 2013 at 9:46 AM, Marco Hugentobler <span dir="ltr"><<a href="mailto:marco.hugentobler@sourcepole.ch" target="_blank">marco.hugentobler@sourcepole.ch</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Hi Willem<br>
<br>
Another thing to explore is the offline editing plugin. It caches
postgis layers to a local spatialite database and can even be used
to synchronise edits between local spatialite db and remote
postgis db.<div><br>
<br>
>Sourcepole have developed (for NIWA, New Zealand) a plugin
that supports local caching of WMS & WFS data sources. This
could probably be adapted to support other input >formats.
Marco Hugentobler did the coding for this - he can perhaps
comment?<br>
<br></div>
The plugin saves WMS as local rasters and WFS as local vectors.
This is similar to save as local vector / raster file by hand.
However the plugin is more convenient because it manages the local
storage and the reference to the remote uri. It has some other
nice features like synchronizing / offlining in a graphical way.
Btw., we hope to include this plugin (a C++ one) into the main
repository after 2.0 feature freeze is over.<br>
<br>
Regards,<br>
Marco <br><div><div>
<br>
On 06.08.2013 03:20, Brent Wood wrote:<br>
</div></div></div>
<blockquote type="cite"><div><div>
<div style="font-size:12pt;font-family:times new roman,new york,times,serif">
<div>Hi Willem,</div>
<div><br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif">Why can you not save such a
layer or a selection from such a layer, as a local shapefile
(or other supported format). is this not effectively a "local
cache"? Delete/overwrite when you desire.<br>
</div>
<div><br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif">Where is the performance
bottleneck? Postgis, network, QGIS? Just creating a local
cache will not necessarily speed things up, if your database
is on a fast server with a fast network, it may even get
slower.<br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif"><br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif">Sourcepole have developed
(for NIWA, New Zealand) a plugin that supports local caching
of WMS & WFS data sources. This could probably be adapted
to support other input formats. Marco Hugentobler did the
coding for this - he can perhaps comment?</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif"><br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif">As Nathan says, caching a
local snapshot of a dynamic, multi-user database will result
in something that gets out of sybc with the source pretty
quickly. Caching a local copy of a largely static database is
relatively safe.</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif"><br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif">Cheers</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif"><br>
</div>
<div style="font-style:normal;font-size:16px;background-color:transparent;font-family:times new roman,new york,times,serif">Brent<br>
</div>
<div style="font-family:times new roman,new york,times,serif;font-size:12pt">
<div style="font-family:times new roman,new york,times,serif;font-size:12pt">
<div dir="ltr">
<hr size="1"> <font face="Arial"> <b><span style="font-weight:bold">From:</span></b> Nathan
Woodrow <a href="mailto:madmanwoo@gmail.com" target="_blank"><madmanwoo@gmail.com></a><br>
<b><span style="font-weight:bold">To:</span></b>
Willem Buitendyk <a href="mailto:willem@pcfish.ca" target="_blank"><willem@pcfish.ca></a> <br>
<b><span style="font-weight:bold">Cc:</span></b>
qgis-user <a href="mailto:qgis-user@lists.osgeo.org" target="_blank"><qgis-user@lists.osgeo.org></a>;
<a href="mailto:qgis-developer@lists.osgeo.org" target="_blank">"qgis-developer@lists.osgeo.org"</a>
<a href="mailto:qgis-developer@lists.osgeo.org" target="_blank"><qgis-developer@lists.osgeo.org></a> <br>
<b><span style="font-weight:bold">Sent:</span></b>
Tuesday, August 6, 2013 11:07 AM<br>
<b><span style="font-weight:bold">Subject:</span></b>
Re: [Qgis-user] Caching remote postgis layers?<br>
</font> </div>
<div><br>
<div>
<div dir="ltr">Hey Willem,
<div><br>
</div>
<div>I'm not aware of a feature like that. The best
place to ask this question to see if anyone is
working, or planning to work, on it is the developer
list. I have copied it in so it show up there too.</div>
<div><br>
</div>
<div>Caching could be done by storing the returned
results in a memory spatialite database. The tricky
thing is knowing when to get any new results from
the postgis database when things have changed on the
database. How does Manifold handling viewing an
area, updating something using SQL on the database
(not though Manifold) and then looking at that area
again?</div>
<div><br>
</div>
<div>- Nathan </div>
</div>
<div><br>
<br>
<div>On Tue, Aug 6,
2013 at 2:44 AM, Willem Buitendyk <span dir="ltr"><<a rel="nofollow" href="mailto:willem@pcfish.ca" target="_blank">willem@pcfish.ca</a>></span>
wrote:<br>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I've noticed looking at
the roadmap that caching remote postgis data is
not on the list. I love qgis but a lot of my work
involves connecting to a remote postgis database.
The speed of navigating a large dataset is, at
the moment, rather unbearable. I also use
Manifold GIS and the difference in speed is
shocking, presumably because Manifold is caching
the dataset. This seems like it should be a
relatively easy thing to implement. Is there
interest on this board for this feature? I was
thinking at taking a stab at contributing to the
source code but am wondering about methods to
achieve data caching most efficiently. Look
forward to others thoughts? Or maybe I missing
something that is already in place?<br>
<br>
Willem<br>
<br>
_______________________________________________<br>
Qgis-user mailing list<br>
<a rel="nofollow" href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
<a rel="nofollow" href="http://lists.osgeo.org/mailman/listinfo/qgis-user" target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/qgis-user" target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
<br>
<br>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><pre>_______________________________________________
Qgis-developer mailing list
<a href="mailto:Qgis-developer@lists.osgeo.org" target="_blank">Qgis-developer@lists.osgeo.org</a>
<a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer" target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><span><font color="#888888">
</font></span></pre><span><font color="#888888">
</font></span></blockquote><span><font color="#888888">
<br>
<br>
<pre cols="72">--
Dr. Marco Hugentobler
Sourcepole - Linux & Open Source Solutions
Weberstrasse 5, CH-8004 Zürich, Switzerland
<a href="mailto:marco.hugentobler@sourcepole.ch" target="_blank">marco.hugentobler@sourcepole.ch</a> <a href="http://www.sourcepole.ch" target="_blank">http://www.sourcepole.ch</a>
Technical Advisor QGIS Project Steering Committee </pre>
</font></span></div>
<br>_______________________________________________<br>
Qgis-developer mailing list<br>
<a href="mailto:Qgis-developer@lists.osgeo.org" target="_blank">Qgis-developer@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer" target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br></div></div><span class="HOEnZb"><font color="#888888"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:'Times New Roman';word-spacing:0px"><span style="font-family:arial;font-size:small"><div>
Angelos Tzotsos</div><div>Remote Sensing Laboratory</div><div>National Technical University of Athens</div><div><a href="http://users.ntua.gr/tzotsos" target="_blank">http://users.ntua.gr/tzotsos</a></div></span></span>
</font></span></div>
<br>_______________________________________________<br>
Qgis-developer mailing list<br>
<a href="mailto:Qgis-developer@lists.osgeo.org">Qgis-developer@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/qgis-developer" target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
<br></blockquote></div><br></div>