[Qgis-user] excessive threads?

[Mike Stanton]

It all depends on the module you are running and whether it effectively uses the “-multi” option with “all_available_cores” and of course how many cores are available on your system.  Yes it will vary a lot depending on the nature of the processing and what stage it is in for that particular module.   Another consideration is the Processor model that is on your system and how many sockets you have active on it. Servers from the so called “Enterprise” class can support 2-4 processor sockets and those processor sockets can have up to 24 cores on them. A Dell R930 (a 4 socket Motherboard) can support are running with 96 cores per system with its current socket, the FC630s blades are capable of 40 cores per system.  The use of the “ALL_AVAILABLE_CORES” option will work on an as needed basis to add cores to the optimum for the workflow modules and the data characteristics being processed.  It gets down to understanding what is normal for your processor model, how much real memory you have, and the nature of the application and the data layout to avoid conflicts.

From: Qgis-user <qgis-user-bounces at lists.osgeo.org> On Behalf Of David Strip via Qgis-user
Sent: Saturday, July 30, 2022 10:44 PM
To: Greg Troxel <gdt at lexort.com>; chris hermansen <clhermansen at gmail.com>
Cc: qgis-user <qgis-user at lists.osgeo.org>
Subject: Re: [Qgis-user] excessive threads?

On 7/30/2022 6:36 PM, Greg Troxel via Qgis-user wrote:

I do understand that threads could help parallelization, eitehr using

multiple cores, or just allowing IO in parallel.



chris hermansen <clhermansen at gmail.com><mailto:clhermansen at gmail.com> writes:



Does it seem like you have one thread per layer for reading plus one for

rendering plus one for user  input?

I don't know how many layers I have :-)    I can experiment with some

smaller projects.



When I noticed this, I had not tried to use qgis in at least a day.




I did a little bit of quick experimentation using Qgis 3.26 (Buenos Aires) on Windows 10. The number of threads is all over the place. It does not seem to be tied to the number of layers in the map, at least in the sense of steady state (ie, let the app sit idle for a while). Adding layers bumps the thread count, with shapefiles adding more threads than files in a geopkg. Locally stored geotiffs are more like a geopkg vector file in terms of threads. But once things settle down I end up around 20 threads. Interestingly, Help->About bumped the thread count by 8. Closing a project (not saving) and then opening a "new" project (no template) bumped another 8 threads for about 30 seconds, then settled back down.Launching Qgis peaked at 30 threads, then after about it minute settles down to 14. I suspect this is dependent on which plug-ins I'm running.

Note these numbers are non-deterministic - Help->About on a fresh start of Qgis with an empty project jumped from 13 threads to 33, then settled down to 22, which is different from the previous attempt where there were open layers in the map.

I'll be interested to hear what drives the thread count to > 150 . None of my very limited test involved database connections or connections to remote data. Those might have something to do with it. Maybe layer filters? Complex rendering (eg, geometry generators)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20220731/07712b5e/attachment-0001.htm>