[pygeoapi] api-first vs web application

Tue Jun 30 14:24:01 PDT 2020

Hi team,
Thanks for raising that discussion Francesco, Jorge, Richard, Tom. To bring some background in the topic I want to share with you some perspectives from where you could look at this topic.

The Web API (developer) perspective, this perspective needs to have a very smooth developer experience, for that reason an API should be small and simple. If the service has multiple audiences/goals, it potentially increases the complexity of client development, scaling and securing the api. My feeling is that most of your concerns are related to this use case. I would therefore like to ask you to bring forward in more detail the challenges that you consider. Because having clear the challenges in advance will also help us to prevent creating the problematic aspects in our software. To me the challenges are not obvious, I would love to learn on this aspect, in order to make a good evaluation.

The traditional spatial analyst is used to opening the traditional OGC W*S services in QGIS as map layers, and will try to engage with OGC API in the same way. He may however be challenged with the increased linkage between features and the hierarchical nature of OGC API Features. In this scenario opening a html view of a feature could actually make sense. We're slowly moving to a spatial data ecosystem not based on the map layer paradigm.

From the URI perspective every item created on the web has a unique URI, preferably exposing its content in various encodings managed by content negotiation. This URI is the identifier for the object on the web. That perspective applied to pygeoapi could mean two things: We make sure every feature exposed via pygeoapi references a unique URI in an alternative application, indicating the source of that feature. Or pygeoapi is the application that 'mints' the uri for the feature. Actually i think we need both approaches, because in some scenario's datasets are re-published with pygeoapi and the source still lives elsewhere on the web. In our Geo4web research (referenced elsewhere) we indicated as a success factor to make the 'landingpage' of a feature as attractive as possible, e.g. allow users to privide feedback on the item, link it to relevant observations, legislations, etc. Links between items is the add-on value created on the web. Minting URI's is a big opportunity for pygeoapi. 

The non OGC aware data analyst finds (assets in) the pygeoapi application (via a search engine or a scientific article). That analyst may not have other tools then the browser and excel. Some basic filter & download capabilities make sense for this use case. 

Wikipedia may want to reference the official location of an item, and may link to a pygeoapi feature as the official source of the location of the item described.

Some alternative considerations:
- With an advanced html UI anybody can test the implementation of the various dataproviders in pygeoapi without any additional tools.
- Having a UI will guide junior developers in understanding the use cases of pygeoapi

Below I'm capturing some of the gitter comments

Francesco: @/all I’d like to raise a discussion on how UI is happening in pygeoapi. I have the impression that we are going to diverge more and more from a web api system. I understood from the spec that the html format would have been just a different view of the same functionalities offered by the same web resources/URIs with content returned in JSON format but now I have the impression this is becoming less true. So is pygeoapi turning into a web application? Is it a risk? How to mitigate it possibly?

Tom Kralidis: IMHO we should have a baseline capability for HTML without turning pygeoapi into a web app, and allow for templating extension by whoever wants/needs.

Jorge: a bit inline with @francbartoli and @tomkralidis HTML output format should be the minimal for compliance, always find that overloading the templates with extra functionality is a dangerous path, it gives a better look and fell but less webapi. I suggest that UI should be less integrated on pygeoapi and more on its own (maybe pygeoapi should have extra REST end points to support a more independent UI), and maybe even have a paralled project pygeoapi-ui to tackle the UI. My 2 cent of an extremely devalued dollar

Paul: Thank you for raising that @francbartoli, it's a valid point of discussion, I'm in favor of good user experience, if that is managed by a secondary application, i'm all fine, but if not, then pygeoapi should provide it.
we should consider that a substantial part of the users arrives to a collection or item page from a search engine, it should then be obvious to that user how he can navigate the collection, querying the collection is one of the aspects, pygeoapi can provide it, but we can also display a statement that the user should use qgis/geonode to query the api
I'm curious to know what are your concerns of moving pygeoapi to be more a web application, one reason i heard about is that a web application has to conform to accessibility guidelines, which an api may not. I'm a bit puzzled by this, not providing a html encoding seems the only way to prevent this
to me the main goal of the html encoding is to bring the world of geodata closer to the larger audience (non developers), and indeed, it brings all sorts of interesting new challenges and use cases
this is one of the aspects Clemens, Lieke and me tried to cover here https://geo4web-testbed.github.io/topic4/#h.82m7wlpx09r6
Also curious to hear if you consider pygeoapi (ogcapi in general) a serious competitor to register applications like geonode, geonetwork, ckan? For some of the use cases, it will certainly be...

Francesco: Thanks Paul for the link, it is certainly useful to the discussion. I’m not against html and search engines in general but I have a strong opinion on the scope of a web api system that imho should focus on basic concepts like being stateful and giving the same user experience regardless of the encoding format used for the content of a specific resource or URI. I don’t consider pygeoapi a competitor of geonode and others rather an ogcapi base building block for more complex web applications. As an example I would see complex UIs on top of it like single page applications like MapStore. Also, despite the cases of crawlers and Internet search engine, I have seen a lot of use cases where pygeoapi is just a cloud native middleware behind an API gateway to be scaled as much as possible and handling the html format is only painful. Turning it into a web application is just breaking the concept of making it a microservice possibly, with several implications from an architectural point of view.

Paul: single page applications such as mapstore, oskari, mapbender, shogun are relevant in some use cases, and i can see a role for pygeoapi to facilitate in those use cases. But there are so many more use cases. For sure I love pygeoapi from the scaling/micro service perspective, so I'm the last to indicate we should leave that... And yes, pygeoapi will be a building block in bigger architectures, sparql engines will for example be able to reason over the data available in pygeoapi, via its json-ld output. I'm just indicating that in all that we should not forget about the simple human being arriving at a pygeoapi-collection, we should provide the necessary context and (links to) tools to perform basic operations. I think we need to discuss what a "web application" is, what negative aspects people consider when thinking about web applications, and try to prevent them.
now a bit more phylosofical... in the end what we're doing here is creating new entities on the web, that link to other entities, and in that process add knowledge to the system.

Francesco: I do believe the main focus for pygeoapi should remain that of an api-first building block for the digital transformation of SDIs. In other words, giving the minimal context to the user in the html form is fair enough for me but deviating from the api-first approach is just an anti-pattern. This wouldn’t exclude to have a pygeoapi-ui project for a web application to cover complex use cases. It’s not a negative aspect but web application and api-first concepts just collide in my opinion

Richard: I honestly don't see the conflict that is being suggested. An HTML representation is not (and must not be, in this case) somehow mutually exclusive with other encoding formats. It's really just the human-first interface to the API. As such, I think the richer this interface is, the better (keeping in mind good UI/UX practice so it doesn't become bloated). I'd be happy if it were split out into a separate repository and/or project, but I don't think pygeoapi has yet reached the maintainability limit of what can be achieved with Jinja templating. I've been working on a way to translate the OGC API - Processing JSON input/output interface into a generic HTML form, and it gets harder the further I take it (e.g. file submission, fluid progress bars). But the principle that there should be an HTML form that allows people to point-and-click to manage geoprocessing jobs seems really valuable to me. Most of the people that I have in mind as consumers of these services are biostatisticians, archivists, etc. who often have no idea what QGIS is. If all they need is a web browser, they don't even have to be aware of what an API is, and they can be successful users. There are some things to keep in mind (e.g. preference for server-side rendering, for the benefit of search engines), but in general I think attention to a good, functional web UI is paramount for the widespread adoption of something modern like pygeoapi.