[QGIS-Developer] Monitor QGIS (Server) memory consumption per project

[kimaidou]

Hi all,

We (@3liz.com) use QGIS Server since a long time to provide hosting
services for our clients, with Lizmap Web Client (LWC) as the web map
builder.

Having hundred LWC instances, we pay obviously attention to metrics, and
monitor the CPU, memory, requests response time, etc.

We sometimes encounter heavy memory consumption for specific projects, for
example :
https://github.com/qgis/QGIS/issues/43787
which put a lot of strain to the servers.

It is often very hard to find the project(s) responsible for a high memory
usage, in a production context, when many requests are run in parallel for
many projects.

I created a POC processing script to audit all the QGIS projects stored in
a folder:
https://gist.github.com/mdouchin/cd89a259d3560635d4bb8708ad430caa/
which helps a bit to gather some key figures, such as the number of layers,
the sum of the images used in map layouts, etc.
This can help, but often the memory consumption is not related to the
number of layers, and we use a "find a needle in the haystack" approach to
find the problematic project(s).

For example, I spent once a whole day finding that a too big JPEG used in a
print layout causes the heavy memory usage:
* look at the memory curve,
* find the moment when the memory increased a lot
* use the command line find  to get which QGIS project files has changed
during this period (for example because it has been uploaded by the user)
* look inside QGIS logs to find which project has been loaded during this
period, and compare
* if you are lucky, there are not so many projects involved...
* download the project, open it in QGIS Desktop to observe and confirm the
memory issue
* try to understand which causes the issue
* ...

We are also building a REST API over py-qgis-server (
https://github.com/3liz/py-qgis-server ) which will allow to get
information on the loaded projects, to ease the auditing process in
real-time. (It will be opensource).

In the processing script, and in this API tool, we still miss a very
important metric: how much each project consumes the system memory (RAM). I
am not sure it could be easily done, but it will be very handy to:

* find heavy memory consumer projects
* see how the memory evolves
* compare the memory usage for each QGIS versions (with the same project
and layers)
* help to find the possible memory leaks

This message is not a feature request, I would like to get your experience
and opinion regarding this memory usage issue. Any contribution welcome ;-)

Thanks in advance
Michaël
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210728/b83dbb3f/attachment.html>