[mapserver-users] mapcache seed slow

[Sebastiano Laini]

I don't know how fast is but I have a whole dedicated server just for do that with a development URL that I'm using just to develop, is still configured in HTTP/1, looking to make some test on a new server to setup HTTP/2 with NGINX and possibly WMTS.

I'm seeding with a bash file and I can see (with TOP) that it's using about 60/80% CPU usage and other HTTPD requests make it from the APACHE user.

Sebastiano Laini
Web Developer

From: Rahkonen Jukka (MML) [mailto:jukka.rahkonen at maanmittauslaitos.fi]
Sent: 22 February 2019 15:03
To: Sebastiano Laini <Sebastiano.Laini at BuchananComputing.co.uk>; 'mapserver-users at lists.osgeo.org' <mapserver-users at lists.osgeo.org>
Subject: Re: mapcache seed slow

Hi,

Stupid question, but do you know how fast your WMS service is? How many GetMap requests/sec it can serve with the metatile size that you are seeding with? Have you some evidence that WMS is running idle while MapCache is busy with cutting the tiles and filling the cache?

-Jukka Rahkonen-

Lähettäjä: mapserver-users <mapserver-users-bounces at lists.osgeo.org<mailto:mapserver-users-bounces at lists.osgeo.org>> Puolesta Sebastiano Laini
Lähetetty: perjantai 22. helmikuuta 2019 13.35
Vastaanottaja: 'mapserver-users at lists.osgeo.org' <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>>
Aihe: [mapserver-users] mapcache seed slow

Hi all,

I'm trying to seed the whole UK at all zoom lv with these resolutions

<resolutions>280 140 70 28 14 7 6.16 2.8 1.4 0.7</resolutions>

But after 13 days it arrives at the 5th level where before I kill the process it seed just 900MB on that level.

This is my script

su -s /bin/sh apache -c "/usr/local/bin/mapcache_seed -c /usr/local/src/mapcache-1.6.1/build/mapcache.xml -t bccache -g bcgrid -z 5,6 -n 400" 2>&1>  /var/www/vhosts/$SITE/scripts/cache_log.txt

I know that in the documentation say that -n should be less or max the number of cpu in the server but is really slow if I put 2, so I tried with 100 and after 13 days that was the result so now I'm trying 400

Is there something else I can use to speed up the seed? I'm using the SQlite cache method

Should I use the -p (| -nprocesses: number of parallel processes that should be used to request tiles from the WMS source) Instead of the -n parameter?

Sebastiano Laini
Web Developer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20190222/0f1fe7f7/attachment.html>