[Gdal-dev] WCS Client Driver

Wed Oct 4 13:36:51 EDT 2006

Hi Frank,

very interesting!

I assume this will be available through Mapserver as well?

Best regards,
Bart

Frank Warmerdam schreef:
> Folks,
>
> I have received client funding for a new driver for GDAL, with
> GDAL acting as a WCS client.  The proposal (less the pricing)
> is included below.  Normally, I'd just launch into something like
> this and announce it upon completion, but it occurs to me that
> perhaps I should be requesting permission from PSC to add a new
> driver since it represents "Adding a substantial amount of new
> code" per RFC 1.  I'm not sure this needs to be written as an RFC
> though.
>
> So, I hereby request permission from the GDAL PSC to implement the
> below described read-only WCS client driver.  Feedback on technical
> details is also welcome.
>
>
>
>     WCS Client Driver for GDAL
>     ==========================
>
> Purpose
> =======
>
> To implement a new GDAL driver for accessing a remote WCS coverage
> in a manner analygous to any local raster dataset.
>
>
> Technical Details
> =================
>
> Overall
> -------
>
> The WCS coverage driver will read a service description file to establish
> the parameters it requires to make GetCoverage calls on a WCS server.  If
> the the service description file lacks DescribeCoverage information for
> the coverage, the driver will invoke DescribeCoverage, and fill in the
> service description file on disk.
>
> Specification Features
> ----------------------
>
> Version:
>  o The driver will support WCS version 1.0.0.
>  o The driver will support WCS version 1.1.0 based on current draft. 
> (Phase 2)
>  o The driver will *not* support WCS version 0.7.
>
> Encodings:
>  o The driver will support GET keyword/value requests.
>  o The driver will *not* support POST XML requests.
>
> Georeferencing:
>  o The driver will support coordinate systems defined only in "EPSG:xyz",
>    "AUTO:xyz" or "OGC:xyz" formats, not GML CRS encodings.
>  o The driver will support georeferencing information for grids only if
>    it is presented as a RectifiedGrid element in the DescribeCoverage.
>
> NullValues:
>  o The driver will support only one "singleValue" null values.  It will
>    not support multiple singleValues, nor intervals.
>
> SupportedFormats:
>  o The driver will support images returned in GeoTIFF, DTED and NITF
>    formats.  Other formats will also be supported if GDAL drivers exist
>    (such as HDF-EOS if GDAL is configured with HDF-EOS format support).
>    At this time GML coverage responses will *not* be supported.
>  o The driver will support multi-part mime bundles as a mechanism to
>    return multiple files (such as JPEG+world file).  (Phase 2)
>
> Interpolation:
>  o The driver will default to use of nearest neighbour interpolation.
>
> Time/Parameter Support:
>  o Supported via overrides in the Service Description File.
>
>
> Service Description File
> ------------------------
>
> The service description file is an XML format file that can hold all the
> information GDAL requires to access a WCS coverage, as well as various
> optional items that can be used to control particular behaviors.
>
> It may include:
>
>  1) The service URL - the base url to which DescribeCoverage and
>     GetCoverage requests is made.
>
>  2) The result of the DescribeCoverage request for the coverage.  This
>     includes the spatial extents, and resolution.
>
>  3) Override values for various parameters, including interpolation,
>     and request format.
>
>  4) The pixel data type for the coverage.
>
>  5) Request values for time and other parameter dimensions.
>
>  6) The coordinate system in Well Known Text format (overrides coordinate
>     system information in the DescribeCoverage).
>
>  7) The geotransform for the coverage (overrides values in the
>     DescribeCoverage).
>
>  8) A preferred block size, and other reading strategy information that
>     may affect performance.
>
>
> Remote Access Strategies
> ------------------------
>
> Getting respectable performance for client applications accessing remote
> WCS servers will in large part depend on the approach taken to turning
> local requests into remote requests.  In particular, because some GDAL
> client applications take a fine grained approach to requesting data from
> drivers, it can be very helpful to aggregate several requests based on
> some blocking scheme.  But other applications may make single large
> read requests of exactly all the data required.  It is desirable to 
> handle
> these as a single request to the remote WCS server that pull back just
> the desired data.
>
> Because the of diversity of situations several read strategies will be
> supported.
>
> 1) Block oriented with caching.  All requests to the remote server will
> made based on the block size, and the blocks will be cached normally in
> the GDAL memory block cache.  The default block size should be 
> substantial,
> perhaps 2MB blocks the width of the image.
>
> 2) 1:1 RasterIO() to GetCoverage. Each RasterIO() request will result 
> in a
> remote GetCoverage request at the resolution and for the exact region
> requested.  Because of the irregular nature of requests, caching is
> impractical.
>
> 3) Hybrid.  Block oriented with caching will be used for smallish 
> requests,
> while very large RasterIO() requests will be done as an immediate call 
> but
> expanded to block boundaries and the result pushed into the block 
> cache.  The
> benefit is that several blocks may be handled in a single request.
>
> The default operation will be to use the Hybrid mode.  Specific 
> strategies
> and block sizes can be forced via the service description file.
>
> Overviews
> ---------
>
> WCS servers will be represented to the client application as having power
> of 2 reduced resolution overview layers.  When using block oriented IO,
> requests will be made at reduced resolutions based on the overview levels
> when appropriate.
>
> Other Issues
> ------------
>
> 1) The DescribeCoverage information does not include any clue as to
> the actual pixel data type of data.  In order for GDALOpen() to return
> for a WCS based on a service description file that contains only the url,
> it will be necessary to perform a DescribeCoverage(), and also to fetch
> a small sample file to establish the pixel data type.
>
> 2) GDAL currently supports opening several file formats from "in memory"
> buffers.  Other file formats do not support this.  As part of this 
> project,
> the driver will first attempt to open the returned datasets using the
> in-memory mechanism.  If that fails, the data will be written to a 
> temporary
> file on disk for access.  Suitable logic for placing temporary files will
> be needed.
>
> 3) The WCS client capability will need to be able to do remote HTTP 
> requests.
> I have used the curl open source library (http://curl.haxx.se/libcurl)
> in several projects very successfully, and would plan to use it for GDAL
> WCS as well.  It will be a requirement for building the WCS support.
>
>
> Testing
> =======
>
> The WCS client will be tested against several WCS servers, including
> hopefully UMN MapServer, IONIC Red Spider, Cubewerx, and PCI Geomatics
> WCS servers.  The GDAL "autotest" suite will also be extended with test
> scripts for the WCS client driver operating against stable public WCS
> servers.
>
>
> Documentation
> =============
>
> Standard "user" documentation will be written for the WCS driver in HTML
> format.  It will include a detailed specification for the service 
> description
> file, and all it's parameters.
>
>
> Intellectual Property
> =====================
>
> The WCS driver will become a regular part of the GDAL library, and it's
> source code and documentation will be released under the normal GDAL
> open source license.
>
>
> Timetable
> =========
>
> Phase 1 (essentially WCS 1.0.0) will be completed within 5 weeks of
> project approval.
>
> Phase 2 (WCS 1.1.0) will be completed within 7 weeks of phase 1 
> completion
> if approved.
>
> Best regards,

-- 
Bart van den Eijnden
OSGIS, Open Source GIS
http://www.osgis.nl