[QGIS-Developer] Preview job and slow datasources

Patrick Dunford enzedrailmaps at gmail.com
Fri Nov 17 02:14:19 PST 2017 

I understand (especially with the other comments about the functionality 
and having to wait for images to be retrieved from servers) what you 
guys are trying to achieve with this feature, but in my project the 
rasters I have loaded cover hundreds of square kilometres, and Qgis 
caches all of them, regardless of how far they are from the particular 
point where I am working on a map.

On 17/11/17 21:56, Alessandro Pasotti wrote:
>
> Maybe a crazy idea, but if this proves to be an issue, what about a 
> monitoring process that prompts the user to disable this feature when 
> memory goes over a certain threshold?
>
> On Fri, Nov 17, 2017 at 8:21 AM, Patrick Dunford 
> <enzedrailmaps at gmail.com <mailto:enzedrailmaps at gmail.com>> wrote:
>
>     Most of them are geoJpeg, there are a smaller number of geoTiff
>
>
>     On 17/11/17 20:10, Matthias Kuhn wrote:
>>
>>     Thanks, that's an interesting information.
>>
>>     What datatype are your rasters?
>>
>>
>>     On 11/17/2017 03:53 AM, Patrick Dunford wrote:
>>>
>>>     With the second command
>>>
>>>     iface.mapCanvas().setPreviewJobsEnabled(False)
>>>
>>>     when I run that command it drops back to 16 GB
>>>
>>>
>>>     On 16/11/17 23:15, Matthias Kuhn wrote:
>>>>
>>>>     Hi Patrick,
>>>>
>>>>     Would be interesting to do this test with previews enabled and
>>>>     disabled. Then we'll see if it's actually the previews or some
>>>>     other mechanism that is causing this.
>>>>
>>>>     IIRC only a composite image is saved in the preview jobs and
>>>>     not each layer separately, but that's just what I remember and
>>>>     not evidence based.
>>>>
>>>>     You can enable them in the python console with
>>>>
>>>>         iface.mapCanvas().setPreviewJobsEnabled(True)
>>>>
>>>>     and disable with
>>>>
>>>>         iface.mapCanvas().setPreviewJobsEnabled(False)
>>>>
>>>>     Thanks for a feedback
>>>>
>>>>     Matthias
>>>>
>>>>
>>>>     On 11/16/2017 08:42 AM, Patrick Dunford wrote:
>>>>>
>>>>>     In my current project using Qgis 2.99, turning off the rasters
>>>>>     uses 16 GB, turning them on uses 52 GB
>>>>>
>>>>>     I do not believe it is independent from the number of layers
>>>>>     that are being displayed.
>>>>>
>>>>>     Previous versions of the software did not cache every single
>>>>>     raster (the number of rasters actually being displayed on the
>>>>>     canvas at any one time is a small fraction of the total number
>>>>>     in the project)
>>>>>
>>>>>     On 16/11/17 20:11, Matthias Kuhn wrote:
>>>>>>
>>>>>>     Hi Patrick,
>>>>>>
>>>>>>     This uses some memory (~ canvas width pixels * canvas height
>>>>>>     pixels * 8 preview images * 32 bit RGBA), so let's assume 50
>>>>>>     MB to 100 MB.
>>>>>>
>>>>>>     This consumption is independent from the number or type of
>>>>>>     layers.
>>>>>>
>>>>>>     Matthias
>>>>>>
>>>>>>
>>>>>>     On 11/16/2017 08:02 AM, Patrick Dunford wrote:
>>>>>>>
>>>>>>>     So to put it another way this is the reason why Qgis wants
>>>>>>>     to use a huge amount of memory (40 GB) when I have a lot of
>>>>>>>     raster images loaded in the background.
>>>>>>>
>>>>>>>
>>>>>>>     On 16/11/17 18:56, Tim Sutton wrote:
>>>>>>>>     Hi
>>>>>>>>
>>>>>>>>>     On 16 Nov 2017, at 04:35, Patrick Dunford
>>>>>>>>>     <enzedrailmaps at gmail.com <mailto:enzedrailmaps at gmail.com>>
>>>>>>>>>     wrote:
>>>>>>>>>
>>>>>>>>>     What is this "preview job" function?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>     Its application logic to prefetch / pre-render offscreen
>>>>>>>>     content in anticipation of user panning the map.
>>>>>>>>
>>>>>>>>     Regards
>>>>>>>>
>>>>>>>>     Tim
>>>>>>>>>
>>>>>>>>>     _______________________________________________
>>>>>>>>>     QGIS-Developer mailing list
>>>>>>>>>     QGIS-Developer at lists.osgeo.org
>>>>>>>>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>>>>>>>>     List info:
>>>>>>>>>     https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>>>>>     Unsubscribe:
>>>>>>>>>     https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>>>>
>>>>>>>>     —
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>     *Tim Sutton*
>>>>>>>>
>>>>>>>>     *Co-founder:*Kartoza
>>>>>>>>     *Project chair:*QGIS.org <http://QGIS.org>
>>>>>>>>
>>>>>>>>     Visit http://kartoza.com <http://kartoza.com/> to find out
>>>>>>>>     about open source:
>>>>>>>>
>>>>>>>>     Desktop GIS programming services
>>>>>>>>     Geospatial web development
>>>>>>>>     GIS Training
>>>>>>>>     Consulting Services
>>>>>>>>
>>>>>>>>     *Skype*: timlinux
>>>>>>>>     *IRC:*timlinux on #qgis at freenode.net <http://freenode.net>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>     _______________________________________________
>>>>>>>     QGIS-Developer mailing list
>>>>>>>     QGIS-Developer at lists.osgeo.org
>>>>>>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>>>>>>     List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>>>     Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>>
>>>>>>
>>>>>>
>>>>>>     _______________________________________________
>>>>>>     QGIS-Developer mailing list
>>>>>>     QGIS-Developer at lists.osgeo.org
>>>>>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>>>>>     List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>>     Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>
>>>>>
>>>>>
>>>>>     _______________________________________________
>>>>>     QGIS-Developer mailing list
>>>>>     QGIS-Developer at lists.osgeo.org
>>>>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>>>>     List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>>     Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     QGIS-Developer mailing list
>>>>     QGIS-Developer at lists.osgeo.org
>>>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>>>     List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>>     Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>
>>>
>>>
>>>     _______________________________________________
>>>     QGIS-Developer mailing list
>>>     QGIS-Developer at lists.osgeo.org
>>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>>     List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>>     Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>
>>
>>
>>     _______________________________________________
>>     QGIS-Developer mailing list
>>     QGIS-Developer at lists.osgeo.org
>>     <mailto:QGIS-Developer at lists.osgeo.org>
>>     List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>>     Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
>
>     _______________________________________________
>     QGIS-Developer mailing list
>     QGIS-Developer at lists.osgeo.org <mailto:QGIS-Developer at lists.osgeo.org>
>     List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>     Unsubscribe:
>     https://lists.osgeo.org/mailman/listinfo/qgis-developer
>     <https://lists.osgeo.org/mailman/listinfo/qgis-developer>
>
>
>
>
> -- 
> Alessandro Pasotti
> w3: www.itopen.it <http://www.itopen.it>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20171117/641e3b49/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: KartozaNewLogoThumbnail.jpg
Type: image/jpeg
Size: 6122 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20171117/641e3b49/attachment-0001.jpg>

More information about the QGIS-Developer mailing list