[QGIS-Developer] Preview job and slow datasources

[Even Rouault]

On vendredi 17 novembre 2017 23:14:19 CET Patrick Dunford wrote:
> I understand (especially with the other comments about the functionality
> and having to wait for images to be retrieved from servers) what you
> guys are trying to achieve with this feature, but in my project the
> rasters I have loaded cover hundreds of square kilometres, and Qgis
> caches all of them, regardless of how far they are from the particular
> point where I am working on a map.
> 

If your rasters are in the same projection and same resolution, did you consider building a 
virtual .VRT catalog from them ? (or group them in catalogs per projection and resolution). 
That could reduce the number of layers.

> On 17/11/17 21:56, Alessandro Pasotti wrote:
> > Maybe a crazy idea, but if this proves to be an issue, what about a
> > monitoring process that prompts the user to disable this feature when
> > memory goes over a certain threshold?
> > 
> > On Fri, Nov 17, 2017 at 8:21 AM, Patrick Dunford
> > 
> > <enzedrailmaps at gmail.com <mailto:enzedrailmaps at gmail.com>> wrote:
> >     Most of them are geoJpeg, there are a smaller number of geoTiff
> >     
> >     On 17/11/17 20:10, Matthias Kuhn wrote:
> >>     Thanks, that's an interesting information.
> >>     
> >>     What datatype are your rasters?
> >>     
> >>     On 11/17/2017 03:53 AM, Patrick Dunford wrote:
> >>>     With the second command
> >>>     
> >>>     iface.mapCanvas().setPreviewJobsEnabled(False)
> >>>     
> >>>     when I run that command it drops back to 16 GB
> >>>     
> >>>     On 16/11/17 23:15, Matthias Kuhn wrote:
> >>>>     Hi Patrick,
> >>>>     
> >>>>     Would be interesting to do this test with previews enabled and
> >>>>     disabled. Then we'll see if it's actually the previews or some
> >>>>     other mechanism that is causing this.
> >>>>     
> >>>>     IIRC only a composite image is saved in the preview jobs and
> >>>>     not each layer separately, but that's just what I remember and
> >>>>     not evidence based.
> >>>>     
> >>>>     You can enable them in the python console with
> >>>>     
> >>>>         iface.mapCanvas().setPreviewJobsEnabled(True)
> >>>>     
> >>>>     and disable with
> >>>>     
> >>>>         iface.mapCanvas().setPreviewJobsEnabled(False)
> >>>>     
> >>>>     Thanks for a feedback
> >>>>     
> >>>>     Matthias
> >>>>     
> >>>>     On 11/16/2017 08:42 AM, Patrick Dunford wrote:
> >>>>>     In my current project using Qgis 2.99, turning off the rasters
> >>>>>     uses 16 GB, turning them on uses 52 GB
> >>>>>     
> >>>>>     I do not believe it is independent from the number of layers
> >>>>>     that are being displayed.
> >>>>>     
> >>>>>     Previous versions of the software did not cache every single
> >>>>>     raster (the number of rasters actually being displayed on the
> >>>>>     canvas at any one time is a small fraction of the total number
> >>>>>     in the project)
> >>>>>     
> >>>>>     On 16/11/17 20:11, Matthias Kuhn wrote:
> >>>>>>     Hi Patrick,
> >>>>>>     
> >>>>>>     This uses some memory (~ canvas width pixels * canvas height
> >>>>>>     pixels * 8 preview images * 32 bit RGBA), so let's assume 50
> >>>>>>     MB to 100 MB.
> >>>>>>     
> >>>>>>     This consumption is independent from the number or type of
> >>>>>>     layers.
> >>>>>>     
> >>>>>>     Matthias
> >>>>>>     
> >>>>>>     On 11/16/2017 08:02 AM, Patrick Dunford wrote:
> >>>>>>>     So to put it another way this is the reason why Qgis wants
> >>>>>>>     to use a huge amount of memory (40 GB) when I have a lot of
> >>>>>>>     raster images loaded in the background.
> >>>>>>>     
> >>>>>>>     On 16/11/17 18:56, Tim Sutton wrote:
> >>>>>>>>     Hi
> >>>>>>>>     
> >>>>>>>>>     On 16 Nov 2017, at 04:35, Patrick Dunford
> >>>>>>>>>     <enzedrailmaps at gmail.com <mailto:enzedrailmaps at gmail.com>>
> >>>>>>>>>     wrote:
> >>>>>>>>>     
> >>>>>>>>>     What is this "preview job" function?
> >>>>>>>>     
> >>>>>>>>     Its application logic to prefetch / pre-render offscreen
> >>>>>>>>     content in anticipation of user panning the map.
> >>>>>>>>     
> >>>>>>>>     Regards
> >>>>>>>>     
> >>>>>>>>     Tim
> >>>>>>>>     
> >>>>>>>>>     _______________________________________________
> >>>>>>>>>     QGIS-Developer mailing list
> >>>>>>>>>     QGIS-Developer at lists.osgeo.org
> >>>>>>>>>     <mailto:QGIS-Developer at lists.osgeo.org>
> >>>>>>>>>     List info:
> >>>>>>>>>     https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >>>>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
> >>>>>>>>>     Unsubscribe:
> >>>>>>>>>     https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >>>>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
> >>>>>>>>     
> >>>>>>>>     —
> >>>>>>>>     
> >>>>>>>>     
> >>>>>>>>     
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20171117/6e6feee6/attachment-0001.html>