[OpenLayers-Dev] ESRI and OpenLayers

[Christopher Schmidt]

Eric,

Your email led me into a response that is really far more appropriate to
the entire dev list in my opinion. ESRI and OpenLayers is a particularly
large hole in the library support, and I wanted to really get down a lot
of my thoughts on why this is. Please take all this with a large grain
of salt as being my personal point of view... but perhaps not an
entirely-inaccurate one, given my level of involvement in the project. 

On Mon, Mar 02, 2009 at 04:27:21PM -0700, Eric Wolf wrote:
> Christopher,
> 
> I hate to open old wounds, but I'm preparing a presentation centered
> around generating a tile cache consumable by OpenLayers but based on
> an ArcGIS map layout. I've tried some different configurations:

So, I want to start off by saying this: No existing OpenLayers core
contributor has, in any major way, a personal or professional motivation
towards supporting ESRI's web services, so far as I am aware.  This
includes the OpenGeo community (generally speaking, tightly coupled with
GeoServer), the Camptocamp group (which works almost exclusively outside
the realm of ESRI, as far as I've observed: possibly motivated by
Europe's lack of proprietary buy-in, hooray), or the DM Solutions
(primarily around MapGuide, and MapServer) or MapGears (primarily around
MapServer) contributors. I expect that if you came with $$ to any of
these groups, they would be likely to be able to help you, but it's not
so common of a problem to make it worthwhile for them to spend time
maintaining ESRI-specific code without a customer asking for it.

With that in mind, almost all work on ESRI-related developments thus far
has been by non-core contributors to the project, and OpenLayers has,
in some cases, not had a great track record with non-core contributors
of content doing the work to get their code in trunk. 

The ESRI AGS code developed thus far has been prey to this: several
people have used it, but not:
 
 * Documented it
 * Tested it
 * Created examples for it

As such, the code is not something that the OpenLayers community can
support without a strong leader stepping up and taking on the task on
the code.

Some counter-examples to this are:

 * The recently committed ArcGIS 9.3 REST API layer.
 * The in-review ESRI ArcXML layer.

Both of these sets of code have had committed developers who have
written tests, provided documented examples, etc. in clean, well-laid
out patches.

Based on your problem description on the mailing list -- something you
neglected to include in this email, which makes it difficult to respond
to precisely -- I can not describe what the best solution is for you,
based on my personal knowledge. I don't use any ESRI products or
anything else.

Though I don't know if it's the best thing for your use case, I do feel
that it would be beneficial to the OpenLayers community to have access
to ESRI's map caches. Cached map tiles are clearly much faster than
non-cached map tiles, and the recent developments of ESRI's -- making
available, for example, sampleserver.esri.com or whatever it is -- make
this type of development much simpler.

However, to accomplish this goal, there still needs to be a champion of
the task -- someone who is reasonably experienced in Javascript, and
willing to step up and take on the task of providing the OpenLayers
community with the support it needs to get this support into the
library. This means that, for example, documentation and examples would
need to be written, etc.  and the contributo would likely need to
demonstrate some willingness to continue to support developers in
maintaining the code, since none of us are, as far as i know,
particularly well-versed or highly motivated to maintain the code at a
reasonable level of quality on our own.

You proposed a number of possible solutions:

> * Turning on WMS in ArcServer and pointing TileCache at it. Works but
> ArcServer wants to serve the layers independently, so I get back to
> the SLD problem.

To me, this seems like a configuation issue. That is, the WMS serves
layers individually, but they're typically possible to combined, with
something like:

layers=0,1,2

In your TileCache config. However, I only know very little of ESRI's
software, so I can't comment on that for sure, and I haven't seen any
links that help me look at your setup to confirm-or-dis-confirm this
notion of mine.  

> * Generating a tile cache out of ArcServer and kludging TileCache to
> serve it up.

I don't know what 'ArcServer' is, but I expect that this would have
essentially the same problem that has existed i ArcGIS caches in the
past, which is that most people who have access to caches don't know
what the parameters used to create the cache are.  

> And lastly:
> 
> * Pulling the AGS plugin for OpenLayers out of the sandbox

This code should probably be in trunk, as described above. The fact that
it is not is because no sufficiently motivated Javascript developer has
made it so. (As Mr. Schaub pointed out at one point, these things arent
meant to happen magically; in Open Source, you need to either pay
someone to do it, or do it yourself.) 

> I haven't actually tried the last bit because of some of your emails
> I've found from 2007. It sounds like AGS is a dead-end. Is this true?
> Is there a specific reason?

AGS is not a dead-end, but you probably read emails where I was
expressing my frustration, centered mostly around the stuff above.
Specifically, the same problem as there is in many aspects of the
project has arisen here:
 * Someone writes some code
 * The code is insufficient, poorly documented, or simply 'not done'
 * The author of the code expects it to get in trunk
 * Requests to do more work to make it happen are ignored.

The general assumption in these cases seems to be that if someone writes
some code, it will automatically be in trunk, which is simply
unrealistic. OpenLayers is a high quality code library, and it seems
completely reasonable to me that in order to keep it that way, we have
to maintain some standards. However, this means that some code does not
make it into trunk in a timely manner, and the patch authors feel
shunned or ignored. Although this is almost always not the case, the
emotions there are very frustrating as a developer: the expectation that
patches lead to changes in the library 'for free' is just unfair to the
core development team.

In this case, I was unable to st up the ArcGIS cached layer code to talk
to any server I could find documented on the web. If I an't do it, I
can't imagine that most users can do it: I'm a pretty smart guy in
general. If I cant' pull it off, then it is probably the case that
others can't either, and that means that putting it in trunk is asking
for pain. With that in mind, I saw no significant personal benefit to
working on it, and the code has sat.

My expressed frustration is/was probably unfair. I'm not aware of any
problem with the code as it exists -- except that people are using it,
and not giving back to the community by trying to get it into trunk. I
find that combination extremely unfair, given the amount of work that so
many people put in to make the library possible. I'd love to see some
organizations out there who are using this code really invest in it, and
make the library better for everyone as a result. This could come in the
form of paying developers to work on it, project sponsorship with a
stated request -- though not binding, clearly somewhat motivating -- or
other potential venues. Without any of this kind of support,
organizations -- of which there are several -- using the AGS code are
simply taking from the library, and not giving back. that type of
behavior is frustrating to me, and that has probably come through in
previous emails. 

> And input would be greatly appreciated.

Using ESRI services in OpenLayers needs champions -- people who are
willing to truly support the cause. ArcXML has a champion in the form of
dzwarg, who has been working with me on an ArcXML ticket. ArcGIS rest
layer had a champion in the form of Jeff Adams. AGS cache needs the
same. The community could probably rally up and make this support
happen: lots of people have expressed interest. Open Source isnt all
about things being done for you, and I'd love to see some of the users
of the software really invest in OpenLayers more than they have up to
this point. ESRI-specific services are a great example of this, where
there's a lot of code floating about, but relatively limited support of
it outside of the companies using it or specific use cases, and that's
a great example of what I'd love to see change.

Sorry for using you email as a jumping off point for a 'rant' like this,
but the state of ESRI-in-OpenLayers is an excellent example of what
*feels* like poor community support of library in favor of immediate
needs, and I really like the idea of that changing. (And if I'm
completely ignoring a significant effort in this regard, this is a
perfect oppourtunity for the existing community members whose work I
have been ignoring to step up and tell me how wrong I am!)    

Regards,
-- 
Christopher Schmidt
MetaCarta