[mapserver-users] mapcache seed speed optimization

Sebastiano Laini Sebastiano.Laini at BuchananComputing.co.uk
Fri Nov 8 08:23:17 PST 2019 

Thanks, I'll take a look at it :),

What should be maxsize to seed? Before was 2084 with  8 8 metatiles now is 4096 with 16 16 metatiles

I found some of the problematic layers, some of them take 4 to 6 seconds in TAB files, while the same layer in raster take 40ms, we just need to find another way to style it.

Sebastiano Laini
Web Developer
Buchanan Computing

From: Rahkonen Jukka (MML) [mailto:jukka.rahkonen at maanmittauslaitos.fi]
Sent: 08 November 2019 15:03
To: Sebastiano Laini <Sebastiano.Laini at BuchananComputing.co.uk>; 'mapserver-users at lists.osgeo.org' <mapserver-users at lists.osgeo.org>
Subject: VS: mapcache seed speed optimization

Hi,

You can see examples about how to debug and improve the rendering speed with Mapserver in the presentation of Lars Schylberg https://media.ccc.de/v/bucharest-271-mapserver-optimize-for-performance.

Your maps look pretty and I hope you will find some bottleneck layers so you can make them faster.

-Jukka-

Lähettäjä: Sebastiano Laini <Sebastiano.Laini at BuchananComputing.co.uk<mailto:Sebastiano.Laini at BuchananComputing.co.uk>>
Lähetetty: perjantai 8. marraskuuta 2019 16.30
Vastaanottaja: 'mapserver-users at lists.osgeo.org' <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>>
Kopio: Rahkonen Jukka (MML) <jukka.rahkonen at maanmittauslaitos.fi<mailto:jukka.rahkonen at maanmittauslaitos.fi>>; 'Yves Jacolin' <yves.jacolin at camptocamp.com<mailto:yves.jacolin at camptocamp.com>>
Aihe: RE: mapcache seed speed optimization

My MapServer is on the same server as MapCache and my network is pretty fast.

I was just trying to experiment with the -n and -p value and checking the CPU load and I didn't seen any difference, but now that I read the responses I'm trying -n 6 and the CPU and the RAM is more in use now and seems to seed faster than before. I reach that numbers because the last time I was seeding with -n 10 was incredibly slow so I started to increase the numbers.

Now in HTOP I can actually see 6 mapcache processes and not just 2

Sadly to seed the whole UK in disk cache we run out of inodes, I would like to use SSD.

About our data we use a mix of everything, here the final result https://www.buchananmapping.co.uk/

We use raster, vector and tab files; the reason I can't move out from mapserver are mainly the mapinfo tables (tab), possibly those tab files are the slowest layers
Even though some of the vector layers are very huge, some of them the data is about 21GB.

I'll try to debug layer by layer through MapServer and strip down to the minimum or replace some layers.

Lastly, the last time I've tried to seed the whole UK it took me more than a month and I simply stop it because was unacceptable to seed 7 levels of resolutions 70, 28, 14, 7, 2.8, 1.4, 0.7 that correspond to these scales 2500, 5000, 10000, 25000, 50000, 100000, 250000, 500000

Sebastiano Laini
Web Developer

From: Rahkonen Jukka (MML) [mailto:jukka.rahkonen at maanmittauslaitos.fi]
Sent: 08 November 2019 14:02
To: Sebastiano Laini <Sebastiano.Laini at BuchananComputing.co.uk<mailto:Sebastiano.Laini at BuchananComputing.co.uk>>; 'mapserver-users at lists.osgeo.org' <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>>
Subject: Re: mapcache seed speed optimization

Hi,

When you seed tiles from WMS the slowest component is usually the WMS server. And what the WMS server is spending its time is usually rendering. And what makes rendering slow is usually that you render too much, and sometimes that WMS server gets the vector data too slow because the backend data source is slow, very often because it has poor indexes which do not for suit the queries that WMS server is
sending.

If you have a slow WMS you can't get more speed out of it by bombing it with very many parallel threads. I would have a try if using 2-4, perhaps 8 threads makes seeding faster but if there is no difference I would concentrate on the WMS configuration. BTW. why did you
select to use 20000 threads?

I am not sure why you see just 1 core running on 100% load. I guess that you are now running WMS and MapCache on the same computer. Mapserver program is using only 1 core but http server should start more processes for parallel requests. That CPU load is at 100% is generally a good thing because it tells that CPU is working, instead of waiting data from a slow disk. Another thing is that CPU may be doing something heavy and stupid.

Myself I would start by gathering  some information about how Mapserver WMS is working be setting "DEBUG 2" on each layer or for the whole mapfile. By looking at the rendering times of each layer the slowest layers can be found and then it is time to think about how to make those layers faster.

You wrote that you use SQLite cache so it would be good to know that SQLite database does not support parallel writes https://sqlite.org/lockingv3.html. Seeding with a few parallel threads may increase the speed if the WMS is the slowest part in your chain but when you have saturated the SQLite writing speed adding threads will do no good. If you have optimized your WMS so well that SQLite sets the limits then you should test other cache alternatives. I guess that file cache would be faster to seed from multiple threads.

When is comes to ticket https://github.com/mapserver/mapcache/issues/169 the latest comment is from a Mpaserver developer and their last words "I wouldn't mind helping to fix if there were interested parties..." feel rather polite to me.

-Jukka Rahkonen-




Lähettäjä: mapserver-users <mapserver-users-bounces at lists.osgeo.org<mailto:mapserver-users-bounces at lists.osgeo.org>> Puolesta Sebastiano Laini
Lähetetty: perjantai 8. marraskuuta 2019 14.35
Vastaanottaja: 'mapserver-users at lists.osgeo.org' <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>>
Aihe: [mapserver-users] mapcache seed speed optimization

Hi all,

I'm evaluating new stacks but seems that I'm stuck with mapserver and mapcache due to the input source of our maps, so I'm trying to improve the speed of the mapserver/mapcache stack and what I cannot improve is the seed speed.

The -n and -p parameters seems to be useless.

-n | -nthreads: number of parallel threads that should be used to request tiles from the WMS source. The default is 1, but can be set higher if the WMS server can withstand parallel requests. (As a rule of thumb, the value chosen here should never be much higher than the number of CPUs on the WMS server.)

-p | -nprocesses: number of parallel processes that should be used to request tiles from the WMS source.
I've tried to use -n 20000 and -p 60000 but for some reason the CPU load is max 100% in 1 core, no way to make it use more power and speed up the seed process.
Max memory used is 470MB between Centos 7 (with all the services running) and the seed process.

I've setup apache to use HTT/2 and it make quite a difference when the cache is already seeded but during the seed process doesn't seems to be affected.

My MapServer build is:

./configure \
--with-ogr=/usr/local/bin/gdal-config \
--with-gdal=/usr/local/bin/gdal-config \
--with-wfsclient \
--with-wmsclient \
--with-wfs \
--enable-debug \
--with-curl-config=/usr/bin/curl-config \
--with-proj \
--with-jpeg \
--with-freetype \
--with-postgis=/usr/pgsql-9.3/bin/pg_config \
--with-geos=/usr/local/bin/geos-config

I know there is mapserver 7 and yet I'm using 6 for the test.

While the MapCache is:

cmake   -DCMAKE_PREFIX_PATH="/usr/bin/sqlite3" \
        -DWITH_SQLITE=1 \
        -DWITH_BERKELEY_DB=0 \
        -DWITH_TIFF=0 \
        -DWITH_GEOTIFF=0 \
        -DWITH_FCGI=0 \
        -DWITH_PCRE=0 \
        -DWITH_PIXMAN=1 \
        -DWITH_OGR=1 \
        -DWITH_GEOS=1 \
         ../

MapCache is the latest version and this is my mapcache.xml file, we use sqlite as a cache method

  <cache name="cache_sqlite" type="sqlite3">
  <dbfile>/home/www/html/maps/cache/{tileset}/{grid}/{z}/{x}-{y}.sqlite3</dbfile>
  <xcount>10000</xcount>
  <ycount>10000</ycount>
  <pragma name="max_page_count">1573741823</pragma>
  </cache>

  <tileset name="bccache">
    <source>bcmaps</source>
    <cache>cache_sqlite</cache>
    <grid>bcgrid</grid>
    <format>PNG</format>
    <metatile>16 16</metatile>
    <metabuffer>0</metabuffer>
    <expires>2628000</expires>
  </tileset>

  <grid name="bcgrid">
               <metadata>
               <title>BC custom grid</title>
               </metadata>
               <srs>EPSG:27700</srs>
               <size>256 256</size>
               <extent>0 0 700000 1250000</extent>
               <resolutions>70 28 14 7 2.8 1.4 0.7</resolutions>
  </grid>

  <format name="mypng" type="PNG">
               <compression>best</compression>
  </format>

  <service type="wms" enabled="true">
    <full_wms>assemble</full_wms>
    <resample_mode>bilinear</resample_mode>
    <format>mypng</format>
    <maxsize>4096</maxsize>
  </service>

I've also tried to use -DWITH_MAPSERVER but seems that the integration is not working and they doen't care? https://github.com/mapserver/mapcache/issues/169 this issue was open 2 years ago and still open, not even a reply from the developers.

How can I improve the seed process?
Should I increase the maxsize and metatile to requests more tiles and cache faster?

Regards,

Sebastiano Laini
Web Developer
Buchanan Computing

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20191108/9b58bea4/attachment-0001.html>

More information about the mapserver-users mailing list