[QGIS-Developer] Preview job and slow datasources

Sun Nov 19 12:44:25 PST 2017

I heard these suggestions, but my project has a very short life, so I 
haven't got the time to experiment really.

On 17/11/17 23:38, Even Rouault wrote:
>
> On vendredi 17 novembre 2017 23:14:19 CET Patrick Dunford wrote:
>
> > I understand (especially with the other comments about the functionality
>
> > and having to wait for images to be retrieved from servers) what you
>
> > guys are trying to achieve with this feature, but in my project the
>
> > rasters I have loaded cover hundreds of square kilometres, and Qgis
>
> > caches all of them, regardless of how far they are from the particular
>
> > point where I am working on a map.
>
> >
>
> If your rasters are in the same projection and same resolution, did 
> you consider building a virtual .VRT catalog from them ? (or group 
> them in catalogs per projection and resolution). That could reduce the 
> number of layers.
>
> > On 17/11/17 21:56, Alessandro Pasotti wrote:
>
> > > Maybe a crazy idea, but if this proves to be an issue, what about a
>
> > > monitoring process that prompts the user to disable this feature when
>
> > > memory goes over a certain threshold?
>
> > >
>
> > > On Fri, Nov 17, 2017 at 8:21 AM, Patrick Dunford
>
> > >
>
> > > <enzedrailmaps at gmail.com <mailto:enzedrailmaps at gmail.com>> wrote:
>
> > > Most of them are geoJpeg, there are a smaller number of geoTiff
>
> > >
>
> > > On 17/11/17 20:10, Matthias Kuhn wrote:
>
> > >> Thanks, that's an interesting information.
>
> > >>
>
> > >> What datatype are your rasters?
>
> > >>
>
> > >> On 11/17/2017 03:53 AM, Patrick Dunford wrote:
>
> > >>> With the second command
>
> > >>>
>
> > >>> iface.mapCanvas().setPreviewJobsEnabled(False)
>
> > >>>
>
> > >>> when I run that command it drops back to 16 GB
>
> > >>>
>
> > >>> On 16/11/17 23:15, Matthias Kuhn wrote:
>
> > >>>> Hi Patrick,
>
> > >>>>
>
> > >>>> Would be interesting to do this test with previews enabled and
>
> > >>>> disabled. Then we'll see if it's actually the previews or some
>
> > >>>> other mechanism that is causing this.
>
> > >>>>
>
> > >>>> IIRC only a composite image is saved in the preview jobs and
>
> > >>>> not each layer separately, but that's just what I remember and
>
> > >>>> not evidence based.
>
> > >>>>
>
> > >>>> You can enable them in the python console with
>
> > >>>>
>
> > >>>>     iface.mapCanvas().setPreviewJobsEnabled(True)
>
> > >>>>
>
> > >>>> and disable with
>
> > >>>>
>
> > >>>>     iface.mapCanvas().setPreviewJobsEnabled(False)
>
> > >>>>
>
> > >>>> Thanks for a feedback
>
> > >>>>
>
> > >>>> Matthias
>
> > >>>>
>
> > >>>> On 11/16/2017 08:42 AM, Patrick Dunford wrote:
>
> > >>>>> In my current project using Qgis 2.99, turning off the rasters
>
> > >>>>> uses 16 GB, turning them on uses 52 GB
>
> > >>>>>
>
> > >>>>> I do not believe it is independent from the number of layers
>
> > >>>>> that are being displayed.
>
> > >>>>>
>
> > >>>>> Previous versions of the software did not cache every single
>
> > >>>>> raster (the number of rasters actually being displayed on the
>
> > >>>>> canvas at any one time is a small fraction of the total number
>
> > >>>>> in the project)
>
> > >>>>>
>
> > >>>>> On 16/11/17 20:11, Matthias Kuhn wrote:
>
> > >>>>>> Hi Patrick,
>
> > >>>>>>
>
> > >>>>>> This uses some memory (~ canvas width pixels * canvas height
>
> > >>>>>> pixels * 8 preview images * 32 bit RGBA), so let's assume 50
>
> > >>>>>> MB to 100 MB.
>
> > >>>>>>
>
> > >>>>>> This consumption is independent from the number or type of
>
> > >>>>>> layers.
>
> > >>>>>>
>
> > >>>>>> Matthias
>
> > >>>>>>
>
> > >>>>>> On 11/16/2017 08:02 AM, Patrick Dunford wrote:
>
> > >>>>>>> So to put it another way this is the reason why Qgis wants
>
> > >>>>>>> to use a huge amount of memory (40 GB) when I have a lot of
>
> > >>>>>>> raster images loaded in the background.
>
> > >>>>>>>
>
> > >>>>>>> On 16/11/17 18:56, Tim Sutton wrote:
>
> > >>>>>>>> Hi
>
> > >>>>>>>>
>
> > >>>>>>>>> On 16 Nov 2017, at 04:35, Patrick Dunford
>
> > >>>>>>>>> <enzedrailmaps at gmail.com <mailto:enzedrailmaps at gmail.com>>
>
> > >>>>>>>>> wrote:
>
> > >>>>>>>>>
>
> > >>>>>>>>> What is this "preview job" function?
>
> > >>>>>>>>
>
> > >>>>>>>> Its application logic to prefetch / pre-render offscreen
>
> > >>>>>>>> content in anticipation of user panning the map.
>
> > >>>>>>>>
>
> > >>>>>>>> Regards
>
> > >>>>>>>>
>
> > >>>>>>>> Tim
>
> > >>>>>>>>
>
> > >>>>>>>>> _______________________________________________
>
> > >>>>>>>>> QGIS-Developer mailing list
>
> > >>>>>>>>> QGIS-Developer at lists.osgeo.org
>
> > >>>>>>>>> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >>>>>>>>> List info:
>
> > >>>>>>>>> https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>>>>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>>>>>> Unsubscribe:
>
> > >>>>>>>>> https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>>>>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>>>>>
>
> > >>>>>>>> —
>
> > >>>>>>>>
>
> > >>>>>>>>
>
> > >>>>>>>>
>
> > >>>>>>>>
>
> > >>>>>>>>
>
> > >>>>>>>>
>
> > >>>>>>>> *Tim Sutton*
>
> > >>>>>>>>
>
> > >>>>>>>> *Co-founder:*Kartoza
>
> > >>>>>>>> *Project chair:*QGIS.org <http://QGIS.org>
>
> > >>>>>>>>
>
> > >>>>>>>> Visit http://kartoza.com <http://kartoza.com/> to find out
>
> > >>>>>>>> about open source:
>
> > >>>>>>>>
>
> > >>>>>>>> Desktop GIS programming services
>
> > >>>>>>>> Geospatial web development
>
> > >>>>>>>> GIS Training
>
> > >>>>>>>> Consulting Services
>
> > >>>>>>>>
>
> > >>>>>>>> *Skype*: timlinux
>
> > >>>>>>>> *IRC:*timlinux on #qgis at freenode.net <http://freenode.net>
>
> > >>>>>>>
>
> > >>>>>>> _______________________________________________
>
> > >>>>>>> QGIS-Developer mailing list
>
> > >>>>>>> QGIS-Developer at lists.osgeo.org
>
> > >>>>>>> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >>>>>>> List
>
> > >>>>>>> info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>>>> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-develo
>
> > >>>>>>> per
>
> > >>>>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>>>
>
> > >>>>>> _______________________________________________
>
> > >>>>>> QGIS-Developer mailing list
>
> > >>>>>> QGIS-Developer at lists.osgeo.org
>
> > >>>>>> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >>>>>> List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>>> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-develop
>
> > >>>>>> er
>
> > >>>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>>
>
> > >>>>> _______________________________________________
>
> > >>>>> QGIS-Developer mailing list
>
> > >>>>> QGIS-Developer at lists.osgeo.org
>
> > >>>>> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >>>>> List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-develope
>
> > >>>>> r
>
> > >>>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>>
>
> > >>>> _______________________________________________
>
> > >>>> QGIS-Developer mailing list
>
> > >>>> QGIS-Developer at lists.osgeo.org
>
> > >>>> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >>>> List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>>
>
> > >>> _______________________________________________
>
> > >>> QGIS-Developer mailing list
>
> > >>> QGIS-Developer at lists.osgeo.org
>
> > >>> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >>> List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >>> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >>
>
> > >> _______________________________________________
>
> > >> QGIS-Developer mailing list
>
> > >> QGIS-Developer at lists.osgeo.org
>
> > >> <mailto:QGIS-Developer at lists.osgeo.org>
>
> > >> List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > >> <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > >
>
> > > _______________________________________________
>
> > > QGIS-Developer mailing list
>
> > > QGIS-Developer at lists.osgeo.org <mailto:QGIS-Developer at lists.osgeo.org>
>
> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > > <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> > > Unsubscribe:
>
> > > https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> > > <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
> -- 
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20171120/fdfb54f8/attachment-0001.html>