[OSGeo-Discuss] State of OpenAerialMap

[Christopher Schmidt]

(This message is CCed to several lists; please reply *only* to the
OpenAerialMap talk list. The list requires subscription. Subscription
information is at:
http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org. Thank
you.)  

State of the OpenAerialMap
==========================

For entirely too long, I've been putting off this email. I apologize for
that.

OpenAerialMap started as a project about a year ago, when I was working
with some aerial data provided by a number of sources. Instead of putting
together all of the different sources, I was interested in combining them
into one world-coverage basemap, to act as an "open" replacement for the
Google Maps imagery layer for purposes where the highest resolutions are
less important, or other data can fill in the gaps.

In part, this project was also started with the idea that it woul be able
to take on a supporting role in the hosting of small datasets, like those
taken via kite photography projects and drone-flown imagery.

In the beginning, the idea was simple: take any data that was available on
the web, convert it to uncompressed geotiffs  in EPSG:4326, spit out some
tiles, and call it done. 

We started with support for GeoTIFFs, and expanded into supporting both
WMS and GDAL tileindexes. The site gained some traction, and started
getting people from different places around the globe -- peopel who had no
idea of how to work with raster data -- attempting to contribute.

>From about February of this year, the situation continued to get worse.
With no active community around the project -- in part due to technical
limitations, in part due to any active groups seeking to help build that
community -- the project stagnated. With the easy problems solved, only
the hard problems were left, and the hard problems tended to be hard to
solve. 

Some of the issues that have been brought up repeatedly:

 * The 'base' dataset for most of the world is a landsat dataset provided
   by i-Cubed. The licensing of this dataset is unclear. It was provided
   to... someone. The usage restrictions on it were not made clear. It is
   possible that this dataset is free to use... or possible that it's not
   at all free to use.
  
   Because this dataset is the base for most of the world -- and appears
   to be a relatively well-served landsat dataset, something generally
   rather rare -- most questions about OpenAerialMap come down to the
   rights of use of this layer.

   There are clearly several paths to solving this particular problem.
   The first path that I've been trying since OAM started was to actually
   get some kind of statement about the rights attached to this layer.
   Unfortunately, that path has failed miserably so far, so it seems that
   it is probably best to abandon this path, and treat the current landsat
   imagery served via OAM as encumbered with copyright restrictions.

   This leaves a second path available, which is to build a landsat
   service which is not encumbered. I believe this should be possible to
   do, and I'm sure that the group of people interested in seeing a
   free/open OAM will be glad to step up.

 * The process of adding data to OAM was, practically speaking, impossible
   for anyone who was not an expert to follow. There are a number of
   technical reason for this, but I think that it is completely reasonable
   to say that these problems can only be resolved by participation in the
   project from people who are at least vaguely knowledgable about working
   with large geographic data.

   I would suggest that what the project really needs is a way to upload
   data to be processed by the OpenAerialMap server itself, rather than
   depending on users to upload 'appropriate' data. This means developing
   a processing queue to convert data from any available format to one
   more suitable for use in OAM. The discussion of what is 'appopriate'
   for OAM requires further discussion as well.

   Combined with this should be improvements to the TileCache, which
   is currently never expired.

 * Storing a whole world sized dataset at the maximum resolution available
   for the whole world -- even at a single given point in time, ignoring
   all other constraints -- is a *really big* 'problem' to solve. One of
   the things that caused my work on loading data into OAM to slow down --
   beyond a lack of interest from any other participants -- is not that
   there was too little data, but instead that there was too much. There
   is 1M or 2M data available for the *entire US* taken every year. This
   data is gigantic -- and it's only a small portion of the total data out
   there.  This one, I don't have a real solution for. I don't see the
   path out of it with any resources that OAM currnetly has available or
   on offer.
  
 * Related to the above, some of the technical possibilities  -- 
   specifically, compressing the data using lossy compression -- has
   issues that are difficult to make happen; these could be improved with
   some work on improving existing libraries, but the work has not been
   done yet. 

 * The current server status is unclear to me. The primary thing that OAM
   has always needed is "disk, more disk!" thus far. Early on, the server
   that OAM runs on got access to a large amount of disk. However, after a
   couple months of loading data, I was told I was using "too much" disk.
   When asking another server admin, I was told to use as much as I
   wanted.

   Clearly, there is some lack of clear understanding about the way that
   OAM should be using its resources. TelaScience, which has acted as a
   gracious host thus far, is not in a position where adminning servers is
   part of their day to day work.

   Clarifying this aspect of things with TelaScience -- what resources are
   available, and what resources should be conserved -- would help allay a
   lot of my personal confusion acting as a host. This also applies with
   regard to other aspects of the actual maintenance of the server.

These are the problems that have existed so far, from my point of view. 

The lack of community around OAM makes all of these problems much more
difficult to handle -- previous cries for help have gone essentially
unnoticed. 

In an effort to kick-start the community, I think we have the following
needs:

 * Improving the relationship with TelaScience, and the servers provided
   by them, to be more clear. 

 * A clear vision for what OAM should be should be defined. 

 * An active community needs to be built around supporting and improving 
   the site, so that it does not stagnate again. 

 * It is important to make it as easy as possible for the target users 
   of OAM to upload data.

I apologize for my poor stewardship of the OpenAerialMap project up to
now, and in more than one case, turning away possible contributors due to
my frustrations at working with the project. At this point, I think it
would be best -- once we have a clearly defined relationship with
TelaScience -- for someone other than me to take over as point person for
communications about the OAM project. 

I would like to suggest that OAM falls within what I consider to me the
responsibilities of the OSGeo Geodata committee, and it probably makes a
lot of sense to stop treating OAM as anything other than an 'arm' of 
that committee.

I hope this helps to clarify the situation with regard to OAM, and spur
the community onwards towards taking the base that has been started with
and developing a real solution to some of the hard problems the project
needs to solve to continue to grow. 

Best Regards,
-- 
Christopher Schmidt