[mapserver-users] Next link missing WFS 2.0 getfeature response for specific bbox

Wed May 12 05:49:39 PDT 2021

Hi all,

I am running into a problem with WFS 2.0 paged GetFeature request. The problem is that for a particular boundingbox a series of paged WFS 2.0 GetFeature requests fails to retrieve all the features. The geometry type of the layer is of `MULTILINESTRING`, the datasource is a GeoPackage file. Running on version 7.6.2.

The problem is that the second paged response (https://geodata.nationaalgeoregister.nl/rioned/gwsw/wfs/v1_0?SERVICE=WFS&SERVICE=WFS&version=2.0.0&service=wfs&request=Getfeature&bbox=80034.6,452005.1,81976.8,453965.8&outputformat=gml3&srsname=EPSG%3A28992&typename=gwsw%3Abeheer_leiding&STARTINDEX=1000 ) is missing the next link. The missing next link does actually produces the expected features (https://geodata.nationaalgeoregister.nl/rioned/gwsw/wfs/v1_0?SERVICE=WFS&SERVICE=WFS&version=2.0.0&service=wfs&request=Getfeature&bbox=80034.6,452005.1,81976.8,453965.8&outputformat=gml3&srsname=EPSG%3A28992&typename=gwsw%3Abeheer_leiding&STARTINDEX=2000 ) and contains a next link itself.

In the MapServer logs, when requesting the page with the missing next link, I see the following log output: `msOGRFileNextShape: Returning MS_DONE (no more shapes)`. This log output also shows when requesting the last page of results, of a paged WFS request.

The interesting thing is that this behaviour does not occur when requesting a slightly bigger bbox: https://geodata.nationaalgeoregister.nl/rioned/gwsw/wfs/v1_0?SERVICE=WFS&SERVICE=WFS&version=2.0.0&service=wfs&request=Getfeature&bbox=80034.6%2C452005.1%2C81976.8%2C453965.8&outputformat=gml3&srsname=EPSG%3A28992&typename=gwsw%3Abeheer_leiding

The SQL query used by Mapserver when requesting the page (https://geodata.nationaalgeoregister.nl/rioned/gwsw/wfs/v1_0?SERVICE=WFS&SERVICE=WFS&version=2.0.0&service=wfs&request=Getfeature&bbox=80034.6,452005.1,81976.8,453965.8&outputformat=gml3&srsname=EPSG%3A28992&typename=gwsw%3Abeheer_leiding&STARTINDEX=1000 ) with the missing next link is:

```
SELECT "beheer_leiding"."uri" AS "uri", "beheer_leiding"."geometry" AS "geometry" FROM "beheer_leiding" JOIN "rtree_beheer_leiding_geometry" ms_spat_idx ON "beheer_leiding".ROWID = ms_spat_idx.id AND ms_spat_idx.minx <= 81976.8 AND ms_spat_idx.maxx >= 80034.6 AND ms_spat_idx.miny <= 453965.8 AND ms_spat_idx.maxy >= 452005.1  ORDER BY "beheer_leiding".ROWID LIMIT 1001 OFFSET 1000
```

This query actually produces 1001 features as expected (other paged requests with more features to come have the same result). So not sure why MapServer decides there are no more results left. The MS_DONE value is returned from here<https://github.com/MapServer/MapServer/blob/dfdda8a18c69f22806c7d6e46bb2bf59f67ed941/mapogr.cpp#L2818> (7.6.2 release) in the MapServer source code.

The problem seems to be caused by an interaction between the data, the requested bbox  and a (by me) suspected bug in MapServer. In particul in the code that determines if there are any features left in the result set. Or maybe I am missing something obvious and there is a mistake in the mapfile causing the issue. Did anyone see this behaviour before? Btw when changing the datasource to a GeoJSON file this behaviour does not occur, but it does occur with a PostGIS datasource.

I created a repo (https://github.com/arbakker/debug-ms-bbox-issue) with a minimal example to reproduce the issue. The repo contains the mapfile, docker-compose, test script, data and a README describing steps in more detail to reproduce the issue.

Cheers,

Anton Bakker
Software Engineer

IT | Geo- en Vastgoedinformatie en Advies |PDOK
06-29 73 73 51 | anton.bakker at kadaster.nl<mailto:anton.bakker at kadaster.nl>

[cid:image001.png at 01D7473E.086BB5A0]<http://www.kadaster.nl/>
[cid:image002.png at 01D7473E.086BB5A0] [cid:image003.png at 01D7473E.086BB5A0] <http://www.linkedin.com/company/kadaster> [cid:image004.png at 01D7473E.086BB5A0] <http://www.twitter.com/het_kadaster> [cid:image005.png at 01D7473E.086BB5A0] <http://www.facebook.com/HetKadaster/> [cid:image006.png at 01D7473E.086BB5A0] <http://www.youtube.com/user/HetKadaster> [cid:image007.png at 01D7473E.086BB5A0] <http://www.instagram.com/kadaster.nl/>

Disclaimer:
De inhoud van dit bericht is uitsluitend bestemd voor geadresseerde.
Gebruik van de inhoud van dit bericht door anderen zonder toestemming van het Kadaster
is onrechtmatig. Mocht dit bericht ten onrechte bij u terecht komen, dan verzoeken wij u
dit direct te melden aan de verzender en het bericht te vernietigen.
Aan de inhoud van dit bericht kunnen geen rechten worden ontleend.

Disclaimer:
The content of this message is meant to be received by the addressee only.
Use of the content of this message by anyone other than the addressee without the consent
of the Kadaster is unlawful. If you have received this message, but are not the addressee,
please contact the sender immediately and destroy the message.
No rights can be derived from the content of this message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 9195 bytes
Desc: image001.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 2315 bytes
Desc: image002.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0008.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 1017 bytes
Desc: image003.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0009.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 967 bytes
Desc: image004.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 959 bytes
Desc: image005.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0011.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 1009 bytes
Desc: image006.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0012.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.png
Type: image/png
Size: 1232 bytes
Desc: image007.png
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20210512/47ec7aeb/attachment-0013.png>