[Geodata] State of OpenAerialMap
crschmidt at crschmidt.net
Thu Dec 18 20:02:45 EST 2008
(This message is CCed to several lists; please reply *only* to the
OpenAerialMap talk list. The list requires subscription. Subscription
information is at:
State of the OpenAerialMap
For entirely too long, I've been putting off this email. I apologize for
OpenAerialMap started as a project about a year ago, when I was working
with some aerial data provided by a number of sources. Instead of putting
together all of the different sources, I was interested in combining them
into one world-coverage basemap, to act as an "open" replacement for the
Google Maps imagery layer for purposes where the highest resolutions are
less important, or other data can fill in the gaps.
In part, this project was also started with the idea that it woul be able
to take on a supporting role in the hosting of small datasets, like those
taken via kite photography projects and drone-flown imagery.
In the beginning, the idea was simple: take any data that was available on
the web, convert it to uncompressed geotiffs in EPSG:4326, spit out some
tiles, and call it done.
We started with support for GeoTIFFs, and expanded into supporting both
WMS and GDAL tileindexes. The site gained some traction, and started
getting people from different places around the globe -- peopel who had no
idea of how to work with raster data -- attempting to contribute.
>From about February of this year, the situation continued to get worse.
With no active community around the project -- in part due to technical
limitations, in part due to any active groups seeking to help build that
community -- the project stagnated. With the easy problems solved, only
the hard problems were left, and the hard problems tended to be hard to
Some of the issues that have been brought up repeatedly:
* The 'base' dataset for most of the world is a landsat dataset provided
by i-Cubed. The licensing of this dataset is unclear. It was provided
to... someone. The usage restrictions on it were not made clear. It is
possible that this dataset is free to use... or possible that it's not
at all free to use.
Because this dataset is the base for most of the world -- and appears
to be a relatively well-served landsat dataset, something generally
rather rare -- most questions about OpenAerialMap come down to the
rights of use of this layer.
There are clearly several paths to solving this particular problem.
The first path that I've been trying since OAM started was to actually
get some kind of statement about the rights attached to this layer.
Unfortunately, that path has failed miserably so far, so it seems that
it is probably best to abandon this path, and treat the current landsat
imagery served via OAM as encumbered with copyright restrictions.
This leaves a second path available, which is to build a landsat
service which is not encumbered. I believe this should be possible to
do, and I'm sure that the group of people interested in seeing a
free/open OAM will be glad to step up.
* The process of adding data to OAM was, practically speaking, impossible
for anyone who was not an expert to follow. There are a number of
technical reason for this, but I think that it is completely reasonable
to say that these problems can only be resolved by participation in the
project from people who are at least vaguely knowledgable about working
with large geographic data.
I would suggest that what the project really needs is a way to upload
data to be processed by the OpenAerialMap server itself, rather than
depending on users to upload 'appropriate' data. This means developing
a processing queue to convert data from any available format to one
more suitable for use in OAM. The discussion of what is 'appopriate'
for OAM requires further discussion as well.
Combined with this should be improvements to the TileCache, which
is currently never expired.
* Storing a whole world sized dataset at the maximum resolution available
for the whole world -- even at a single given point in time, ignoring
all other constraints -- is a *really big* 'problem' to solve. One of
the things that caused my work on loading data into OAM to slow down --
beyond a lack of interest from any other participants -- is not that
there was too little data, but instead that there was too much. There
is 1M or 2M data available for the *entire US* taken every year. This
data is gigantic -- and it's only a small portion of the total data out
there. This one, I don't have a real solution for. I don't see the
path out of it with any resources that OAM currnetly has available or
* Related to the above, some of the technical possibilities --
specifically, compressing the data using lossy compression -- has
issues that are difficult to make happen; these could be improved with
some work on improving existing libraries, but the work has not been
* The current server status is unclear to me. The primary thing that OAM
has always needed is "disk, more disk!" thus far. Early on, the server
that OAM runs on got access to a large amount of disk. However, after a
couple months of loading data, I was told I was using "too much" disk.
When asking another server admin, I was told to use as much as I
Clearly, there is some lack of clear understanding about the way that
OAM should be using its resources. TelaScience, which has acted as a
gracious host thus far, is not in a position where adminning servers is
part of their day to day work.
Clarifying this aspect of things with TelaScience -- what resources are
available, and what resources should be conserved -- would help allay a
lot of my personal confusion acting as a host. This also applies with
regard to other aspects of the actual maintenance of the server.
These are the problems that have existed so far, from my point of view.
The lack of community around OAM makes all of these problems much more
difficult to handle -- previous cries for help have gone essentially
In an effort to kick-start the community, I think we have the following
* Improving the relationship with TelaScience, and the servers provided
by them, to be more clear.
* A clear vision for what OAM should be should be defined.
* An active community needs to be built around supporting and improving
the site, so that it does not stagnate again.
* It is important to make it as easy as possible for the target users
of OAM to upload data.
I apologize for my poor stewardship of the OpenAerialMap project up to
now, and in more than one case, turning away possible contributors due to
my frustrations at working with the project. At this point, I think it
would be best -- once we have a clearly defined relationship with
TelaScience -- for someone other than me to take over as point person for
communications about the OAM project.
I would like to suggest that OAM falls within what I consider to me the
responsibilities of the OSGeo Geodata committee, and it probably makes a
lot of sense to stop treating OAM as anything other than an 'arm' of
I hope this helps to clarify the situation with regard to OAM, and spur
the community onwards towards taking the base that has been started with
and developing a real solution to some of the hard problems the project
needs to solve to continue to grow.
More information about the Geodata