[gdal-dev] OAPIF and apikey

[Rahkonen Jukka (MML)]

Even Rouault wrote:

> On mardi 12 mai 2020 21:21:30 CEST Rahkonen Jukka (MML) wrote:
>> Hi,
>> 
>> I wonder how I could deal with an OGC API Features service that requires to
>> use apikey as an extra parameter in each request. First step is easy
>> because I can just add the apikey parameter to the URL of the landing page
>> (?api-key=xxxyyyzzzkkk) but the link to collections in the document that is
>> returned comes without the apikey and ogrinfo and og2ogr are using the link
>> as is. Result is a "HTTP 401 not authorized".
 
> Hum, normally there's code to handle that in the driver. And even a test (against a stub server).
> Are you using GDAL 3.1 / master ? If so, how can I replicate that ?

There was a misconfiguration on the server side yesterday and it did not work. Now the configuration is corrected and apikey is indeed added. That certain server will be opened within a few weeks but extra parameters can be tested with any OAPIF server.
 
ogrinfo oapif:"https://beta-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1?apikey=111222" hylky --debug on
HTTP: Fetch(https://beta-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/hylky/items?limit=10&apikey=111222)

There is a small glitch with GDAL and the new service. On the contrary to what I wrote yesterday, the new service, when it is configured right, adds the api-key that is used for accessing the landing page into the links. Now GDAL is picking the links which contain api-key parameter and adds api-key again from the connection URL. The result looks like this:

HTTP: Fetch(https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/hylky/items?next=MDoxMTY%3D&api-key=12be7903-e203-4836-9bd9-8cc6bdfcf880&limit=10&api-key=12be7903-e203-4836-9bd9-8cc6bdfcf880)

Duplicated api-key does not make trouble for our server because the values are equal. The next test with two duplicated parameters reveals that our server is using the first KVP parameter.

ogrinfo oapif:"https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/?api-key=12be7903-e203-4836-9bd9-8cc6bdfcf880&limit=5" hylky --debug on

The generated requests contain &limit=10 and &limit=5 and value 10 is used.
HTTP: Fetch(https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/hylky/items?next=MDoxMzk%3D&api-key=12be7903-e203-4836-9bd9-8cc6bdfcf880&limit=10&api-key=12be7903-e203-4836-9bd9-8cc6bdfcf880&limit=5)

This leads to next question: Is there a way to run ogrinfo for a small sample of OAPIF collection that can contain millions of features? I can set the page size (limit) with an open option but ogrinfo follows the "next" links until the end of data. I can use ogr2ogr with the -limit option as a quite effective workaround  because ogr2ogr seems to read just as many pages from OAPIF that are needed to reach the desired count.  Ogrinfo with -sql "select * from oapif_layer limit 3" is a poor workaround because filtering happens after the whole collection is fetched. I would rather see something like OAPIF open option like "hard_limit/count/maxfeatures" that sets the standard OAPIF parameter &limit= and does not follow the next links.

Finally, I think that the default page_size could be increased to 1000 from the default that is 10.

-Jukka-