<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
code
{mso-style-priority:99;
font-family:"Courier New";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m evaluating new stacks but seems that I’m stuck with mapserver and mapcache due to the input source of our maps, so I’m trying to improve the speed of the mapserver/mapcache stack and what I cannot improve is the seed speed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The –n and –p parameters seems to be useless.<o:p></o:p></p>
<p style="line-height:18.0pt;background:white"><strong><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#3E4349">-n | –nthreads</span></strong><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#3E4349">: number of parallel threads
that should be used to request tiles from the WMS source. The default is 1, but can be set higher if the WMS server can withstand parallel requests. (As a rule of thumb, the value chosen here should never be much higher than the number of CPUs on the WMS server.)<o:p></o:p></span></p>
<p style="line-height:18.0pt;background:white"><strong><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#3E4349">-p | –nprocesses</span></strong><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#3E4349">: number of parallel
processes that should be used to request tiles from the WMS source.<o:p></o:p></span></p>
<p class="MsoNormal">I’ve tried to use –n 20000 and –p 60000 but for some reason the CPU load is max 100% in 1 core, no way to make it use more power and speed up the seed process.<o:p></o:p></p>
<p class="MsoNormal">Max memory used is 470MB between Centos 7 (with all the services running) and the seed process.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ve setup apache to use HTT/2 and it make quite a difference when the cache is already seeded but during the seed process doesn’t seems to be affected.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My MapServer build is:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">./configure \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-ogr=/usr/local/bin/gdal-config \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-gdal=/usr/local/bin/gdal-config \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-wfsclient \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-wmsclient \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-wfs \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--enable-debug \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-curl-config=/usr/bin/curl-config \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-proj \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-jpeg \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-freetype \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-postgis=/usr/pgsql-9.3/bin/pg_config \<o:p></o:p></span></code></p>
<p class="MsoNormal"><code><span style="font-size:10.0pt">--with-geos=/usr/local/bin/geos-config<o:p></o:p></span></code></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I know there is mapserver 7 and yet I’m using 6 for the test.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">While the MapCache is: <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">cmake -DCMAKE_PREFIX_PATH="/usr/bin/sqlite3" \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_SQLITE=1 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_BERKELEY_DB=0 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_TIFF=0 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_GEOTIFF=0 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_FCGI=0 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_PCRE=0 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_PIXMAN=1 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_OGR=1 \<o:p></o:p></p>
<p class="MsoNormal"> -DWITH_GEOS=1 \<o:p></o:p></p>
<p class="MsoNormal"> ../<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">MapCache is the latest version and this is my mapcache.xml file, we use sqlite as a cache method<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> <cache name="cache_sqlite" type="sqlite3"><o:p></o:p></p>
<p class="MsoNormal"> <dbfile>/home/www/html/maps/cache/{tileset}/{grid}/{z}/{x}-{y}.sqlite3</dbfile><o:p></o:p></p>
<p class="MsoNormal"> <xcount>10000</xcount><o:p></o:p></p>
<p class="MsoNormal"> <ycount>10000</ycount><o:p></o:p></p>
<p class="MsoNormal"> <pragma name="max_page_count">1573741823</pragma><o:p></o:p></p>
<p class="MsoNormal"> </cache><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> <tileset name="bccache"><o:p></o:p></p>
<p class="MsoNormal"> <source>bcmaps</source><o:p></o:p></p>
<p class="MsoNormal"> <cache>cache_sqlite</cache><o:p></o:p></p>
<p class="MsoNormal"> <grid>bcgrid</grid><o:p></o:p></p>
<p class="MsoNormal"> <format>PNG</format><o:p></o:p></p>
<p class="MsoNormal"> <metatile>16 16</metatile><o:p></o:p></p>
<p class="MsoNormal"> <metabuffer>0</metabuffer><o:p></o:p></p>
<p class="MsoNormal"> <expires>2628000</expires><o:p></o:p></p>
<p class="MsoNormal"> </tileset><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> <grid name="bcgrid"><o:p></o:p></p>
<p class="MsoNormal"> <metadata><o:p></o:p></p>
<p class="MsoNormal"> <title>BC custom grid</title><o:p></o:p></p>
<p class="MsoNormal"> </metadata><o:p></o:p></p>
<p class="MsoNormal"> <srs>EPSG:27700</srs><o:p></o:p></p>
<p class="MsoNormal"> <size>256 256</size><o:p></o:p></p>
<p class="MsoNormal"> <extent>0 0 700000 1250000</extent><o:p></o:p></p>
<p class="MsoNormal"> <resolutions>70 28 14 7 2.8 1.4 0.7</resolutions><o:p></o:p></p>
<p class="MsoNormal"> </grid><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> <format name="mypng" type="PNG"><o:p></o:p></p>
<p class="MsoNormal"> <compression>best</compression><o:p></o:p></p>
<p class="MsoNormal"> </format><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> <service type="wms" enabled="true"><o:p></o:p></p>
<p class="MsoNormal"> <full_wms>assemble</full_wms><o:p></o:p></p>
<p class="MsoNormal"> <resample_mode>bilinear</resample_mode><o:p></o:p></p>
<p class="MsoNormal"> <format>mypng</format><o:p></o:p></p>
<p class="MsoNormal"> <maxsize>4096</maxsize><o:p></o:p></p>
<p class="MsoNormal"> </service><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ve also tried to use –DWITH_MAPSERVER but seems that the integration is not working and they doen’t care?
<a href="https://github.com/mapserver/mapcache/issues/169">https://github.com/mapserver/mapcache/issues/169</a> this issue was open 2 years ago and still open, not even a reply from the developers.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">How can I improve the seed process?<o:p></o:p></p>
<p class="MsoNormal">Should I increase the maxsize and metatile to requests more tiles and cache faster?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:EN-GB">Sebastiano Laini<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:EN-GB">Web Developer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:EN-GB">Buchanan Computing<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>