[Qgis-user] [Lizmap] Lizmap/qgis server : cache management (seeding)
celati Laurent
laurent.celati at gmail.com
Wed May 17 03:35:36 PDT 2023
Good morning,
Thanks again for your usefull replies.
I paste 2 screen shot. 1st one showing a relevant data 's display at
1:50000. All is ok at the begining.
[image: image.png]
And below , a 2nd one. After several zoom in /zoom out, we can notice
bottlenecks
regarding the display. it seems that the data display is no longer updated
according to the zoom level.
In the legend, we can notice slowness too.
[image: image.png]
[image: db9c22cd-5d90-4a95-8621-ae7e904a1ba9.png]
I thought was a good idea to customize cache by layer. That 's why i tried
to follow this page
https://docs.lizmap.com/3.5/fr/admin/cache.html
But unfortunatly, i'm faced on the issue (the main/initial message of this
post).
- My lizmap parameters :
Cache :
Files type
Root directory cache :
/tmp/
Cache lifetime :
0
QGIS serveur
Version de QGIS Serveur*
Width max. défaut GetMap
3000
Height max. default GetMap
3000
- Physical server : 16 Go RAM CPU 4 cores
- Qgis Project configuration and layer : 5 light postgis vector tables
(no raster) (hosted on another linux server).
These postgis tables have an spatial index (GIST on geometric field).
Qgis Desktop -> Qgis server tab Services enabled/layers published for
wms,wmts and wfs
- Within lizmap plugin qgis client desktop :
Layers tab : All layers have PNG format with navigateur client cache
with 300s and uniq tile. Server tiles cache disabled (i make some tests
yesterday with this option but i noticed worst performances).
No base map setted/ simplification for the rendering if possible : yes
enabled. Geometries are no complex.
The qgis project in desktop is fluid and fast (opening, moving, zooming).
- Server configuration :
You right. Indeed, when i restart qgis server, i can see enviro
variables enabled, especially cache directory, parallel rendering, max
threads...below, please read my vhost apache :
*<VirtualHost *:80> ServerAdmin webmaster at localhost ServerName
xxxxxxxxxx.fr <http://xxxxxxxxxx.fr> DocumentRoot /var/www/html # Apache
logs (different than QGIS Server log) ErrorLog
${APACHE_LOG_DIR}/xxxxxxxxxx.fr.error.log CustomLog
${APACHE_LOG_DIR}/xxxxxxxxxx.fr.access.log combined # Longer timeout for
WPS... default = 40 FcgidIOTimeout 120 FcgidInitialEnv LC_ALL
"en_US.UTF-8" FcgidInitialEnv PYTHONIOENCODING UTF-8 FcgidInitialEnv LANG
"en_US.UTF-8" # QGIS log FcgidInitialEnv QGIS_SERVER_LOG_STDERR 1
FcgidInitialEnv QGIS_SERVER_LOG_LEVEL 0 FcgidInitialEnv
QGIS_SERVER_LOG_FILE /var/log/qgis/qislog.log # parallel rendering
FcgidInitialEnv QGIS_SERVER_PARALLEL_RENDERING "1" #
QGIS_SERVER_CACHE_DIRECTORY FcgidInitialEnv QGIS_SERVER_CACHE_DIRECTORY
"/tmp/" # default QGIS project SetEnv QGIS_PROJECT_FILE
/home/qgis/projects/world.qgs # QGIS_AUTH_DB_DIR_PATH must lead to a
directory writeable by the Server's FCGI process user FcgidInitialEnv
QGIS_AUTH_DB_DIR_PATH "/home/qgis/qgisserverdb/" FcgidInitialEnv
QGIS_AUTH_PASSWORD_FILE "/home/qgis/qgisserverdb/qgis-auth.db" # Set pg
access via pg_service file SetEnv PGSERVICEFILE
/home/qgis/.pg_service.conf FcgidInitialEnv PGPASSFILE
"/home/qgis/.pgpass" # plugin qgis_server FcgidInitialEnv QGIS_PLUGINPATH
"/var/www/qgis-server/plugins" # lizmap FcgidInitialEnv
QGIS_SERVER_LIZMAP_REVEAL_SETTINGS True # Needed for Qgis HelloServer
plugin HTTP basic auth <IfModule mod_fcgi.c> RewriteEngine on
RewriteCond %{HTTP:Authorization} . RewriteRule .* -
[E=HTTP_AUTHORIZATION:%{HTTP:Authorization}] </IfModule> # if qgis-server
is installed from packages in debian based distros this is usually
/usr/lib/cgi-bin/ # run "locate qgis_mapserv.fcgi" if you don't know where
qgis_mapserv.fcgi is ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory
"/usr/lib/cgi-bin/"> AllowOverride None Options +ExecCGI -MultiViews
-SymLinksIfOwnerMatch Require all granted </Directory> <IfModule
mod_fcgid.c> FcgidMaxRequestLen 26214400 FcgidConnectTimeout 60
</IfModule></VirtualHost>*
Thanks so much !
Le mar. 16 mai 2023 à 12:35, René-Luc Dhont <rldhont at gmail.com> a écrit :
> Hi,
>
> You need more than 1 QGIS Server process running on your server to get a
> responsive Web client.
> I think you have more than 1 QGIS Server process available to response to
> lizmap web client requests to QGIS Server.
> The number of available QGIS Server processes depend on the server you use
> for QGIS Server (apache, nginx, or py-qgis-server) and the interface used
> for QGIS Server (fcgi or py-qgis-server).
>
> Then for the lizmap server cache configuration, it has to be done in the
> Lizmap Desktop plugin for each layer you want to cache on the server.
> You can find the detail here
> https://docs.lizmap.com/current/en/publish/configuration/optimization.html#server-side
>
> Regards,
> René-Luc
>
> Le 16/05/2023 à 12:08, celati Laurent a écrit :
>
> Good morning,
>
> Thanks for your message, i have currently one only Qgis server
> installed/available.
> Sorry but what do you mean by "enhance the number of QGIS Server available
> ?"
>
> Following to my previous message, regarding my initial and main issue, For
> information, when i do :
>
> php lizmap/console.php project:load
>
> i get :
>
> */var/www/lizmap/lizmap-web-client-3.6.3# php lizmap/console.php project:load
> Enter the repository lizmaptest
> Get the project zios
> 1 request(s) return success for the project zios
> Enter the repository aero
> Get the project aero_v2
> 1 request(s) return success for the project aero_v2
> Enter the repository dr0
> Get the project forages
> 1 request(s) return success for the project forages
> Get the project forages_2154
> 1 request(s) return success for the project forages_2154
> Get the project dr0
> 1 request(s) return success for the project dr0
> Get the project forages_2154_with_fond
> 1 request(s) return success for the project forages_2154_with_fond*
>
> *But when i do : *
>
> *php lizmap/console.php wmts:capabilities -v dr0 forages_2154
> No layers configured with cache!*
>
> *Do you know where the problem comes from ?*
>
> In parrarel, always with the aim of improving performance, i tried to edit my vhost apache in adding :
>
> FcgidInitialEnv QGIS_SERVER_PARALLEL_RENDERING "1"
>
> and
>
> FcgidInitialEnv QGIS_SERVER_CACHE_DIRECTORY "/tmp/"
>
> (Is it a good idea or i should rather keep the default value ('cache in profile directory') ?
>
> i setted this path because into my lizmap cache config i noticed : cache root directory : /tmp/)
>
> Still in parralel, I discovered the existence of these 2 qgis server plugins. Would it be adviced to installing them?https://github.com/3liz/qgis-server-wmts-cache-plugin
>
> https://github.com/3liz/qgis-server-tiles-plugin
>
> Eventually, I'm taking the liberty to paste below an update of my qgis server logs :
>
> I'm surprised to not read in the environment summary logs, the new enviro variables 'FcgidInitialEnv QGIS_SERVER_PARALLEL_RENDERING'
> and 'FcgidInitialEnv QGIS_SERVER_CACHE_DIRECTORY'
>
> How to ensure that the cache server is functional and taken into account?
>
> 11:30:43 INFO Server[919778]: ******************** New request ***************
> 11:30:43 INFO Server[919778]: Request URL: http://sig-rec.andra.fr/cgi-bin/qgis_mapserv.fcgi/lizmap/server.json
> 11:30:43 INFO Server[919778]: Environment:
> 11:30:43 INFO Server[919778]: ------------------------------------------------
> 11:30:43 INFO Server[919778]: SERVER_NAME: sig-rec.andra.fr
> 11:30:43 INFO Server[919778]: REQUEST_URI: /cgi-bin/qgis_mapserv.fcgi/lizmap/server.json
> 11:30:43 INFO Server[919778]: SCRIPT_NAME: /cgi-bin/qgis_mapserv.fcgi
> 11:30:43 INFO Server[919778]: PATH_INFO: /lizmap/server.json
> 11:30:43 INFO Server[919778]: REMOTE_ADDR: 192.168.4.7
> 11:30:43 INFO Server[919778]: SERVER_PORT: 80
> 11:30:43 INFO Server[919778]: QUERY_STRING:
> 11:30:43 INFO Server[919778]: REQUEST_METHOD: GET
> 11:30:43 INFO Server[919778]: QGIS_PROJECT_FILE: /home/qgis/projects/world.qgs
> 11:30:43 INFO Server[919778]: SERVER_PROTOCOL: HTTP/1.1
> 11:30:43 INFO Server[919778]: Using configuration file path from environment: /home/qgis/projects/world.qgs
> 11:30:43 INFO Lizmap[919778]: Request headers provided
> 11:30:43 INFO Lizmap[919778]: No lizmap user groups in request headers
> 11:30:43 INFO Server[919778]: Using configuration file path from environment: /home/qgis/projects/world.qgs
> 11:30:43 INFO Server[919778]: Trying URL path: '/cgi-bin/qgis_mapserv.fcgi/lizmap/server.json' for '/lizmap'
> 11:30:43 INFO Server[919778]: API Lizmap accepts the URL path '/cgi-bin/qgis_mapserv.fcgi/lizmap/server.json'
> 11:30:43 INFO Server[919778]: Checking API path /server.json for server.json
> 11:30:43 INFO Server[919778]: API Lizmap: found handler server
> 11:30:43 CRITICAL Server[919778]: Service ATLAS is not registered
> 11:30:43 CRITICAL Server[919778]: Service CADASTRE is not registered
> 11:30:43 INFO Server[919778]: Request finished in 6 ms
>
> Thans a lot again for your usefull help ans support.
>
>
> Le lun. 15 mai 2023 à 15:40, celati Laurent <laurent.celati at gmail.com> a
> écrit :
>
>> Hi, thanks,
>> Probably not. Should i set additional environment variables in my apache
>> vhost ?
>> (MAX_CACHE_LAYERS, QGIS_SERVER_PARALLEL_RENDERING, QGIS_SERVER_MAX_THREADS,
>> QGIS_SERVER_MAX_THREADS,QGIS_SERVER_CACHE_SIZE, QGIS_SERVER_WMS_MAX_HEIGHT
>> / QGIS_SERVER_WMS_MAX_WIDTH)
>>
>> Below, a part of my parameters qgis server log (summary) :
>>
>> 14:10:05 INFO Server[907652]: ******************** New request
>> ***************
>>
>> 14:10:05 INFO Server[907652]: Request URL:
>> http://xxxx-xxx.xxxx.fr/cgi-bin/qgis_mapserv.fcgi?map=%2Fhome%2Fdata%2Fftp%2Ftemplate%2Fqgis%2Flizmap_test%2FDR0%2Fforages_2154.qgs&request=GetCapabilities&service=WMS&version=1.3.0&Lizmap_User=&Lizmap_User_Groups=
>>
>> 14:10:05 INFO Server[907652]: Environment:
>>
>> 14:10:05 INFO Server[907652]:
>> ------------------------------------------------
>>
>> 14:10:05 INFO Server[907652]: SERVER_NAME: xxxxx.xxxx.fr
>>
>> 14:10:05 INFO Server[907652]: REQUEST_URI:
>> /cgi-bin/qgis_mapserv.fcgi?map=%2Fhome%2Fdata%2Fftp%2Ftemplate%2Fqgis%2Flizmap%5Ftest%2FDR0%2Fforages%5F2154%2Eqgs&request=GetCapabilities&service=WMS&version=1%2E3%2E0&Lizmap%5FUser=&Lizmap%5FUser%5FGroups=
>>
>> 14:10:05 INFO Server[907652]: SCRIPT_NAME: /cgi-bin/qgis_mapserv.fcgi
>>
>> 14:10:05 INFO Server[907652]: REMOTE_ADDR: 192.168.4.7
>>
>> 14:10:05 INFO Server[907652]: SERVER_PORT: 80
>>
>> 14:10:05 INFO Server[907652]: QUERY_STRING:
>> map=%2Fhome%2Fdata%2Fftp%2Ftemplate%2Fqgis%2Flizmap%5Ftest%2FDR0%2Fforages%5F2154%2Eqgs&request=GetCapabilities&service=WMS&version=1%2E3%2E0&Lizmap%5FUser=&Lizmap%5FUser%5FGroups=
>>
>> 14:10:05 INFO Server[907652]: REQUEST_METHOD: GET
>>
>> 14:10:05 INFO Server[907652]: QGIS_PROJECT_FILE:
>> /home/qgis/projects/world.qgs
>>
>> 14:10:05 INFO Server[907652]: SERVER_PROTOCOL: HTTP/1.1
>>
>> 14:10:05 INFO Lizmap[907652]: Request headers provided
>>
>> 14:10:05 INFO Lizmap[907652]: No lizmap user groups in request headers
>>
>> 14:10:05 INFO Lizmap[907652]: Lizmap user groups in parameters :
>>
>> 14:10:05 INFO Lizmap[907652]: Fetching
>> /home/data/ftp/template/qgis/lizmap_test/DR0/forages_2154.qgs.cfg cfg file
>> with last modified timestamp : 1684152008.8653688
>>
>> 14:10:05 INFO Lizmap[907652]: No acl defined in Lizmap config
>>
>> 14:10:05 INFO Server[907652]: LIZMAP_USER:
>>
>> 14:10:05 INFO Server[907652]: LIZMAP_USER_GROUPS:
>>
>> 14:10:05 INFO Server[907652]:
>> MAP:/home/data/ftp/template/qgis/lizmap_test/DR0/forages_2154.qgs
>>
>> 14:10:05 INFO Server[907652]: REQUEST:GetCapabilities
>>
>> 14:10:05 INFO Server[907652]: SERVICE:WMS
>>
>> 14:10:05 INFO Server[907652]: VERSION:1.3.0
>>
>> 14:10:05 INFO Lizmap[907652]: Request headers provided
>>
>> 14:10:05 INFO Lizmap[907652]: No lizmap user groups in request headers
>>
>> 14:10:05 INFO Lizmap[907652]: Lizmap user groups in parameters :
>>
>> 14:10:05 INFO Lizmap[907652]: Fetching
>> /home/data/ftp/template/qgis/lizmap_test/DR0/forages_2154.qgs.cfg cfg file
>> with last modified timestamp : 1684152008.8653688
>>
>> 14:10:05 INFO Lizmap[907652]: Request headers provided
>>
>> 14:10:05 INFO Lizmap[907652]: No lizmap user groups in request headers
>>
>> 14:10:05 INFO Lizmap[907652]: Lizmap user groups in parameters :
>>
>> 14:10:05 INFO Lizmap[907652]: Fetching
>> /home/data/ftp/template/qgis/lizmap_test/DR0/forages_2154.qgs.cfg cfg file
>> with last modified timestamp : 1684152008.8653688
>>
>> 14:10:05 WARNING Server[907652]: Found WMS capabilities document in cache
>>
>> 14:10:05 INFO Server[907652]: Request finished in 8 ms
>>
>> Le lun. 15 mai 2023 à 15:11, Giovanni Manghi <
>> giovanni.manghi at naturalgis.pt> a écrit :
>>
>>> Hi,
>>>
>>>
>>> On debian/apache, I succeed in installing qgis server and lizmap. I am a
>>>> little surprised by the poor performance that I can observe currently.
>>>>
>>>
>>>
>>> Have you enabled (QGIS Server) parallel rendering in your Apache/Ngnix
>>> configs?
>>>
>>> -- G --
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230517/2acb77c8/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 102271 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230517/2acb77c8/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 402283 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230517/2acb77c8/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: db9c22cd-5d90-4a95-8621-ae7e904a1ba9.png
Type: image/png
Size: 23242 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230517/2acb77c8/attachment-0005.png>
More information about the QGIS-User
mailing list