[Gdal-dev] WCS Client Driver

Frank Warmerdam warmerdam at pobox.com
Wed Oct 4 13:12:40 EDT 2006 

Folks,

I have received client funding for a new driver for GDAL, with
GDAL acting as a WCS client.  The proposal (less the pricing)
is included below.  Normally, I'd just launch into something like
this and announce it upon completion, but it occurs to me that
perhaps I should be requesting permission from PSC to add a new
driver since it represents "Adding a substantial amount of new
code" per RFC 1.  I'm not sure this needs to be written as an RFC
though.

So, I hereby request permission from the GDAL PSC to implement the
below described read-only WCS client driver.  Feedback on technical
details is also welcome.



	WCS Client Driver for GDAL
	==========================

Purpose
=======

To implement a new GDAL driver for accessing a remote WCS coverage
in a manner analygous to any local raster dataset.


Technical Details
=================

Overall
-------

The WCS coverage driver will read a service description file to establish
the parameters it requires to make GetCoverage calls on a WCS server.  If
the the service description file lacks DescribeCoverage information for
the coverage, the driver will invoke DescribeCoverage, and fill in the
service description file on disk.

Specification Features
----------------------

Version:
  o The driver will support WCS version 1.0.0.
  o The driver will support WCS version 1.1.0 based on current draft. (Phase 2)
  o The driver will *not* support WCS version 0.7.

Encodings:
  o The driver will support GET keyword/value requests.
  o The driver will *not* support POST XML requests.

Georeferencing:
  o The driver will support coordinate systems defined only in "EPSG:xyz",
    "AUTO:xyz" or "OGC:xyz" formats, not GML CRS encodings.
  o The driver will support georeferencing information for grids only if
    it is presented as a RectifiedGrid element in the DescribeCoverage.

NullValues:
  o The driver will support only one "singleValue" null values.  It will
    not support multiple singleValues, nor intervals.

SupportedFormats:
  o The driver will support images returned in GeoTIFF, DTED and NITF
    formats.  Other formats will also be supported if GDAL drivers exist
    (such as HDF-EOS if GDAL is configured with HDF-EOS format support).
    At this time GML coverage responses will *not* be supported.
  o The driver will support multi-part mime bundles as a mechanism to
    return multiple files (such as JPEG+world file).  (Phase 2)

Interpolation:
  o The driver will default to use of nearest neighbour interpolation.

Time/Parameter Support:
  o Supported via overrides in the Service Description File.


Service Description File
------------------------

The service description file is an XML format file that can hold all the
information GDAL requires to access a WCS coverage, as well as various
optional items that can be used to control particular behaviors.

It may include:

  1) The service URL - the base url to which DescribeCoverage and
     GetCoverage requests is made.

  2) The result of the DescribeCoverage request for the coverage.  This
     includes the spatial extents, and resolution.

  3) Override values for various parameters, including interpolation,
     and request format.

  4) The pixel data type for the coverage.

  5) Request values for time and other parameter dimensions.

  6) The coordinate system in Well Known Text format (overrides coordinate
     system information in the DescribeCoverage).

  7) The geotransform for the coverage (overrides values in the
     DescribeCoverage).

  8) A preferred block size, and other reading strategy information that
     may affect performance.


Remote Access Strategies
------------------------

Getting respectable performance for client applications accessing remote
WCS servers will in large part depend on the approach taken to turning
local requests into remote requests.  In particular, because some GDAL
client applications take a fine grained approach to requesting data from
drivers, it can be very helpful to aggregate several requests based on
some blocking scheme.  But other applications may make single large
read requests of exactly all the data required.  It is desirable to handle
these as a single request to the remote WCS server that pull back just
the desired data.

Because the of diversity of situations several read strategies will be
supported.

1) Block oriented with caching.  All requests to the remote server will
made based on the block size, and the blocks will be cached normally in
the GDAL memory block cache.  The default block size should be substantial,
perhaps 2MB blocks the width of the image.

2) 1:1 RasterIO() to GetCoverage. Each RasterIO() request will result in a
remote GetCoverage request at the resolution and for the exact region
requested.  Because of the irregular nature of requests, caching is
impractical.

3) Hybrid.  Block oriented with caching will be used for smallish requests,
while very large RasterIO() requests will be done as an immediate call but
expanded to block boundaries and the result pushed into the block cache.  The
benefit is that several blocks may be handled in a single request.

The default operation will be to use the Hybrid mode.  Specific strategies
and block sizes can be forced via the service description file.

Overviews
---------

WCS servers will be represented to the client application as having power
of 2 reduced resolution overview layers.  When using block oriented IO,
requests will be made at reduced resolutions based on the overview levels
when appropriate.

Other Issues
------------

1) The DescribeCoverage information does not include any clue as to
the actual pixel data type of data.  In order for GDALOpen() to return
for a WCS based on a service description file that contains only the url,
it will be necessary to perform a DescribeCoverage(), and also to fetch
a small sample file to establish the pixel data type.

2) GDAL currently supports opening several file formats from "in memory"
buffers.  Other file formats do not support this.  As part of this project,
the driver will first attempt to open the returned datasets using the
in-memory mechanism.  If that fails, the data will be written to a temporary
file on disk for access.  Suitable logic for placing temporary files will
be needed.

3) The WCS client capability will need to be able to do remote HTTP requests.
I have used the curl open source library (http://curl.haxx.se/libcurl)
in several projects very successfully, and would plan to use it for GDAL
WCS as well.  It will be a requirement for building the WCS support.


Testing
=======

The WCS client will be tested against several WCS servers, including
hopefully UMN MapServer, IONIC Red Spider, Cubewerx, and PCI Geomatics
WCS servers.  The GDAL "autotest" suite will also be extended with test
scripts for the WCS client driver operating against stable public WCS
servers.


Documentation
=============

Standard "user" documentation will be written for the WCS driver in HTML
format.  It will include a detailed specification for the service description
file, and all it's parameters.


Intellectual Property
=====================

The WCS driver will become a regular part of the GDAL library, and it's
source code and documentation will be released under the normal GDAL
open source license.


Timetable
=========

Phase 1 (essentially WCS 1.0.0) will be completed within 5 weeks of
project approval.

Phase 2 (WCS 1.1.0) will be completed within 7 weeks of phase 1 completion
if approved.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org

More information about the Gdal-dev mailing list