[OSGeo-Discuss] The OSGeo response to the proposed "GeoServices REST API" document [was: Would you be concerned ...]
acuster at gmail.com
Thu May 9 09:20:35 PDT 2013
Hey Cameron, all,
Cameron, you recently asked me to join your letter from the OSGeo to the
OGC Members regarding the adoption of the proposed "ESRI GeoServices
REST API" as an OGC standard.
Thanks. Your request has forced me to define my position, which has been
a huge waste of my time, but was probably worth doing.
My response has two sides, my position on the debate and my decision to
participate in the letter.
The first is my position on the actual proposed standard.
* The OGC should be in the business of developing good standards,
not of choosing which standards should be implemented.
* The proposers of the document want to make a standard and have
followed all the rules of the consortium. The work of any such
group of members deserves serious, good faith consideration.
* The need for an integrated suite of services using simple data,
which is addressed (partially) by the document, is real. The
proposed document is pushing the OGC on this issue.
* The proposed document could be useful to a number of people.
* The proposed document is not significantly more broken than the
existing standards of the OGC. As an author of standards at the
OGC, I know how totally impossible it is to write a good standard,
so the weaknesses in the existing document seem more acceptable.
* The OGC is, de-facto, in the position of recommending the
interoperable standards for geospatial services. The proposed
document is not good enough, not widely enough implemented, and
not publicly supported enough, to be considered at the same level
as existing standards.
* Adopting a standard implies a desire to maintain the standard
but the desire to support this approach by the OGC membership is
limited. The lack of collaboration on this version bodes ill for
* The overlap in functionality between the proposed services and the
existing services, notably with the ongoing work to modularize the
existing services, is almost 100 percent. However, compatibility is
* There is already a published document:
so there is no need for the document to be adopted as an
OGC Standard merely for interoperability with the ESRI
* The document, as a new, separate effort, repeats mistakes which were
made and since solved by the other services.
* The document focuses on the past (notably with backwards
compatibility and use of only GET/POST) not on the future.
* The document needs a comprehensive editorial rewrite. (see at end)
The problem for me, is that both simple answers are bad.
A simple acceptance of the standard would introduce a new set of 'OGC
approved' open services. The OGC approval might enable governments to
buy a XXXX-new-name-here-XXXX solution instead of a W*S or a S*S
solution. (On that, I am inclined to let them make their own mistakes.)
The path forwards towards harmonizing the services is unclear. There is
little good will towards the standard on behalf of the membership.
Fixing this document in addition to fixing the W*S services will be a pain.
Simply rejecting the solution would be bad for the OGC. It would place
the OGC in the position of picking winners and losers in the standards
business. It would mean that the OGC is stuck on the project of fixing
the W*S standards to meet some nebulous future functionality without
having any path to get there. It would discourage innovation and progress.
Is there a third way?
Well, actually, there is. The natural consequence of either decision is
a renewed commitment towards trying to build this thing that we want, an
integrated suite of simple services built using simple, well defined
resources, accessible and usable using the core HTTP verbs, and
discoverable through link following. That's the way forwards and why I
have wasted a chunk of my life on WMS-Next.
My personal conclusion, then, is that the vote does not concern me.
I suspect the vote is mostly of interest to the commercial entities,
groups whose self-interest is so destructive everywhere because it does
not place user freedom and action first and foremost. Acceptance will
have several net negative impacts on me but that's life, eh?
For me, the path forwards is clear: make WMS-Next kick ass. The vote is
merely a waste of my time.
The second side to my response regards the actual letter.
* The letter comes from the Free Software community which I think
could have a voice on the matter.
* The letter was started by someone who has my respect.
* The letter re-enforces the link between OSGeo and the OGC.
* The letter expresses many concerns which I share.
* The letter comes from OSGeo, a group with which I have an
ambivalent relationship from being a second class citizen and
from its focus on 'open source' rather than on freedom.
* As a member of the OGC, I have a direct voice there (albeit again
as a second class member without a vote) so it makes more sense
for me to act directly at the OGC than indirectly through OSGeo.
* The letter is only rejection of the proposal without offering an
alternative way forwards.
* The critique of the letter is not yet fully formulated making it
hard to decide if I actually agree with it.
* The critiques do not inspire me. I suspect none of you has actually
read the proposed document, nor have any real analysis to offer of
it. (EDIT: actually, a new wave of criticisms focused on 'imagery'
seem to come from folk who actually have read the document. Cool!)
* Some the concerns are wrong. (It seems, this is simply due to lack
of knowledge of the details, a minor issue ultimately).
My personal conclusion then is that I should not be trying to express
myself on the matter through an incompletely formed statement of the
Therefore, I will not add myself to your letter.
Nonetheless, thanks for your effort, thanks for asking, and thanks for
prodding me to sit down and write this; given the time I have taken, it
is clear that I wanted to make myself do it.
A Quick Critique of the proposed standard document.
1. The name.
The document currently uses a name that is confusing in all discussions
on the matter, both internally at the OGC and externally.
(Yesterday, the Standards Working Group proposing the document agreed to
accept proposals for new names; if they can find a good alternative,
they may adopt it.)
2. The design.
The document does not show an overall, coherent design to the service
suite. This is mostly because the design reflects a suite that evolved
piecemeal at ESRI rather than one designed from the start to meet the
needs of a broad swath of users. This is also due because the experts at
the OGC did not contribute to an improved design for a multitude of reasons.
The design of the service which does exist is focused on yesteryear. As
an apology for the existing design, section~6.2.2 of part 1, Core says
that the only HTTP verbs used are GET and PUT because "the API was
originally developed several years ago" but that's okay because it
enables support for "Microsoft Silverlight" and "Adobe Flex" two
standards which are dying rapidly. In 2013, a new standard ought to be
focused on the next 10 years not the past ten.
3. The writing.
The document is poorly written.
3.1 The document is poorly introduced.
The document should start with a discussion of the overall design of the
suite of services and what each individual service provides within that
suite. This would make it easy for a user to understand the focus of the
suite. (The note between requirements~3 and requirement~4, stating that
the spec makes no requirements on clients, is really part of the scope
of the document and should be front and center.)
The document instead starts with a whole section on REST.
The GeoServices REST API provides a standard way for web clients to
communicate with geographic information system (GIS) servers based
on Representational State Transfer (REST) principles.
The scope claims that the document identifies resources and a way to use
"structured URLs" to exchange those resources between clients and
services. If that were true, I would expect
(1) a list of resources at least as an example, and
(2) an example of the structure and structuring principles of
Section~6 stats this process. Section~6.1 introduces the concept of a
root URL for each service and mixes in that that root URL is also a
catalog and that the hierarchy offers resources. This all happens in a
single, overly compact paragraph. This section needs an editorial
rewrite for clarity separating
(1) the end point,
(2) the catalog,
(3) the URL hierarchy,
(3) the resources, and
(4) interaction of clients with the URLs through the exchange
(get/post) of the resources to the different URLs.
Unfortunately, section~6 then decays into a meandering discussion of
REST which mentions the buzzword a lot but does not discuss the aspects
of REST design that have been met by the suite and those that have not.
If REST is a form of design which offers certain benefits, OGC standards
should be discussing the benefits achieved not the buzz word. To what
extent are the resources cachable and how do clients and servers agree
on that? To what extent are the endpoints and actions on those endpoints
discoverable, and how? Those are the issues of RESTful design.
Section~6.3 discusses Resources but immediately then states:
Each resource has a unique URL. Resources MAY support parameters.
which is a discussion of the URL hierarchy. (And 'Resources' do not
'support parameters'! Services may 'handle requests containing
parameters' if that is what is meant.) Then the standard introduces a
new idea 'Controller Resources' which are able to 'edit' and 'query'!? A
web search does not reveal any formal discussion of 'Controller
Resource' so this new type of resource probably deserves some kind of
Section~6.3.3 is not really about resources but the fact that the URL
hierarchy is not discoverable from the Resources themselves.
Section~6.3.4 does give a list of resources, but since 'image' is not
part of it but will be used by the Service for Map Images, I surmise
that the list is of the things 'we have designed a JSON representation
for' rather than a list of 'the resources identified and exchanged in
the service suite' which is what I expect in an introductory design.
This introduction therefore needs a substantial editorial rewrite to be
effective in its goal of explaining what the suite of standards actually
This criticism is not limited to the core document.
The Service for Maps document also starts without setting out the five
core elements (end pt, catalog?, URL hierarchy, resources, and
Section~6 of the Service for Maps part, in the middle of the second
paragraph, introduces a new concept, with capitalized letters:
The Map Service Root resource includes a tables property ...
without giving us any clue what this is. It "provides basic information"
(perhaps it is a data structure?) but it also "supports several
operations" (maybe it's a magical 'Controller Resource'). To me, this
just reads as 'the authors of the spec are confused and imprecise in
their written language'.
Then, in the figure of 'Resources,' there is actually a resource called
'All layers and tables'. Sorry, but this reader needs some real help to
understand what that resource might be.
3.2 The document requirements are poor.
The very first requirement, Requirement~1 of the Core document, states:
Req 1 If a request uses the HTTP GET method, the processing of the
request by the service SHALL be safe and idempotent as specified
by RFC 2616, 9.1.
This is a *ridiculous* requirement, completely untestable, and
explicitly what the Mod Spec should be preventing with its attempt to
require all injunctive language to be accompanied by formal tests. The
formal test indeed shows that the requirement is untestable.
Inspect the documentation of the implementation to identify, if
requests specified in the GeoServices REST API standard that
support the HTTP GET method, are all safe and idempotent, i.e.
free of side- effects, as specified in HTTP (RFC 2616, section
9.1). Alternatively, ask the developer of the the service.
Note that it is not possible to run automated tests against a
service to verify conformance with the requirement (and IETF
provides no test for conformance with HTTP either). In the absence
of such tests, a statement of the developers of the service is
sufficient indication that the requirement was addressed.
So essentially this requirement is met if the developers say 'yeah, we
thought about it.' In other words, requirement 1 of the whole shebang is
a recommendation. Contrast this with requirement 2 which has a nice test
defined for it and is a good, solid requirement.
Again, broadening the critique to the Services for Maps, its first
Req 1 The Map Service Root resource SHALL accept requests that
conform to the URI template in Table and use any HTTP method
identified in the same table.
but the test is then completely irrelevant. At the protocol level, all
web services 'shall accept' request messages, so we are not discussing
that. A functional test for this 'shall accept' is to construct all
valid requests and see that the service never returns an exception to
those requests. Even worse, the first test is a catch all 'please test
the service here' test. This is *not* a well written specification of
the tests for the injunctions of the Service for Maps.
The requirements need a complete, systematic review to see that they
work, they are testable, and have good tests.
4. The document has numerous other issues which need resolution
* There is no discussion of the overlap of the f= parameter and of the
HTTP format header. What happens if they conflict?
* The lack of full integration of the four core HTTP verbs, notably
DELETE and PUT, exists merely for backwards compatibility not for
* Requirement~3 blocks forwards progress and experimentation with no
explanation as to its importance.
~~~~This goes on and on. This is the work for an EDITOR.
I am not interested in doing that work.
More information about the Discuss