[mapserver-users] Speed of getting layer extents does matter

[Rahkonen Jukka (MML)]

Hi,

I tried to repeat the issue with slow GetCapabilities and my hypothesis was that the reason was in the new nested layer groups which I had added lately. However, now it seems that the real issue was in the way I had added new Spatialite layers. That I were using those layers in groups was not important in my case.

I separated 7 layers from a production service for the speed tests. Three of the data tables have less than 10000 rows, three between 10000and 60000, and one 205000 rows so they are not big tables at all. Initially the time to read GetCapabilities with curl was 4.1 seconds. By rewriting the DATA lines as described below the time went down to 0.45 seconds.

I had followed the examples from http://www.mapserver.org/input/vector/ogr.html without thinking too much about what I was doing:

CONNECTIONTYPE OGR
CONNECTION "db.sqlite"
DATA "select * from my_table"

With this definition the layer extents are computed by aggregating the extents of all the geometries of the table. However, GetCapabilities is much faster if I define the layer as

CONNECTIONTYPE OGR
CONNECTION "db.sqlite"
DATA "my_table"

I have up-to-date statistics in my Spatialite DB and the boost in speed must mean that OGR is clever enough for making a query to the layer_statistics table for getting the layer extents.

Now afterwards this feels trivial but I would suggest to improve the documentation in  http://www.mapserver.org/input/vector/ogr.html and http://mapserver.org/mapfile/layer.html Currently the latter document is putting rather low value on the importance of extents.

EXTENT [minx] [miny] [maxx] [maxy]
The spatial extent of the data. In most cases you will not need to specify this, but it can be used to avoid the speed cost of having MapServer compute the extents of the data.

I am not sure what could be the best place to add some documentation about this speed cost which can easily mean that GetCapabilities gets half a second slower than necessary per ill behaving layer in the mapfile. I sketched this text

Performance considerations
Layer extents and slow WMS GetCapabilities

Computing the extents of an OGR layer can be expensive if the layer is large and extents can't be read directly from metadata. This is the case if:

-          Data source does not support fast method for accessing the extents (CSV, GML)

-          The source store does not have layer statistics available or they are not up-to-date and therefore disabled (Spatialite, PostGIS)

-          Layer data in the mapfile is defined as SQL SELECT

For example, if Spatialite database has a valid entry in the layer_statistics table GDAL/OGR is able to read the extents directly from the metadata when the layer is defined as

CONNECTIONTYPE OGR
CONNECTION "db.sqlite"
DATA "my_table"

However, if layer is defined as SQL SELECT the extents are computed by running a query that aggregates the extents of selected geometries
CONNECTIONTYPE OGR
CONNECTION "db.sqlite"
DATA "select * from my_table"

If computing the layer extents on-the-fly feels too slow consider to use the EXTENT keyword in LAYER http://mapserver.org/mapfile/layer.html.

For comparison, with Geoserver the layer extents are stored to configuration files when layers are added to server for the first time which means the same as having LAYER-EXTENT always set for all the layers in mapfiles.

-Jukka Rahkonen-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20150306/c7e9c217/attachment-0001.html>