<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
span.Shkpostityyli17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 70.85pt 2.0cm;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:836463756;
mso-list-type:hybrid;
mso-list-template-ids:-1775074024 559988210 67829763 67829765 67829761 67829763 67829765 67829761 67829763 67829765;}
@list l0:level1
{mso-level-start-at:97;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:Calibri;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US">I tried to repeat the issue with slow GetCapabilities and my hypothesis was that the reason was in the new nested layer groups which I had added lately. However, now it seems that the real issue was in the way I had added
new Spatialite layers. That I were using those layers in groups was not important in my case.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I separated 7 layers from a production service for the speed tests. Three of the data tables have less than 10000 rows, three between 10000and 60000, and one 205000 rows so they are not big tables at all. Initially the
time to read GetCapabilities with curl was 4.1 seconds. By rewriting the DATA lines as described below the time went down to 0.45 seconds.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><br>
I had followed the examples from <a href="http://www.mapserver.org/input/vector/ogr.html">
http://www.mapserver.org/input/vector/ogr.html</a> without thinking too much about what I was doing:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTIONTYPE OGR<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTION “db.sqlite"<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">DATA "select * from my_table"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">With this definition the layer extents are computed by aggregating the extents of all the geometries of the table. However, GetCapabilities is much faster if I define the layer as <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTIONTYPE OGR<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTION “db.sqlite"<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">DATA "my_table"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I have up-to-date statistics in my Spatialite DB and the boost in speed must mean that OGR is clever enough for making a query to the layer_statistics table for getting the layer extents.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Now afterwards this feels trivial but I would suggest to improve the documentation in <a href="http://www.mapserver.org/input/vector/ogr.html">http://www.mapserver.org/input/vector/ogr.html</a> and
<a href="http://mapserver.org/mapfile/layer.html">http://mapserver.org/mapfile/layer.html</a> Currently the latter document is putting rather low value on the importance of extents.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:FI">EXTENT [minx] [miny] [maxx] [maxy]<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:FI">The spatial extent of the data. In most cases you will not need to specify this, but it can be used to
avoid the speed cost of having MapServer compute the extents of the data.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I am not sure what could be the best place to add some documentation about this speed cost which can easily mean that GetCapabilities gets half a second slower than necessary per ill behaving layer in the mapfile. I sketched
this text<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Performance considerations<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Layer extents and slow WMS GetCapabilities<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Computing the extents of an OGR layer can be expensive if the layer is large and extents can’t be read directly from metadata. This is the case if:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span lang="EN-US"><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US">Data source does not support fast method for accessing the extents (CSV, GML)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span lang="EN-US"><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US">The source store does not have layer statistics available or they are not up-to-date and therefore disabled (Spatialite, PostGIS)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span lang="EN-US"><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US">Layer data in the mapfile is defined as SQL SELECT<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">For example, if Spatialite database has a valid entry in the layer_statistics table GDAL/OGR is able to read the extents directly from the metadata when the layer is defined as<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTIONTYPE OGR<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTION “db.sqlite"<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">DATA "my_table"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">However, if layer is defined as SQL SELECT the extents are computed by running a query that aggregates the extents of selected geometries<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTIONTYPE OGR<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">CONNECTION “db.sqlite"<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:9.0pt;font-family:"Courier New"">DATA "select * from my_table"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">If computing the layer extents on-the-fly feels too slow consider to use the EXTENT keyword in LAYER
<a href="http://mapserver.org/mapfile/layer.html">http://mapserver.org/mapfile/layer.html</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">For comparison, with Geoserver the layer extents are stored to configuration files when layers are added to server for the first time which means the same as having LAYER-EXTENT always set for all the layers in mapfiles.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-Jukka Rahkonen- <o:p></o:p></span></p>
</div>
</body>
</html>