[Gdal-dev] WCS Client Driver

Kralidis,Tom [Burlington] Tom.Kralidis at ec.gc.ca
Wed Oct 4 13:45:37 EDT 2006


Hi,

(jumping up and down -- good news!).

This will undoubtedly increase the usage of WCS servers.

Though I'm not a part of the PSC, I provide the following comments
(interleaved): 

> -----Original Message-----
> From: gdal-dev-bounces at lists.maptools.org 
> [mailto:gdal-dev-bounces at lists.maptools.org] On Behalf Of 
> Frank Warmerdam
> Sent: 04 October, 2006 1:13 PM
> To: gdal-dev
> Subject: [Gdal-dev] WCS Client Driver
> 
> Folks,
> 
> I have received client funding for a new driver for GDAL, 
> with GDAL acting as a WCS client.  The proposal (less the 
> pricing) is included below.  Normally, I'd just launch into 
> something like this and announce it upon completion, but it 
> occurs to me that perhaps I should be requesting permission 
> from PSC to add a new driver since it represents "Adding a 
> substantial amount of new code" per RFC 1.  I'm not sure this 
> needs to be written as an RFC though.
> 
> So, I hereby request permission from the GDAL PSC to 
> implement the below described read-only WCS client driver.  
> Feedback on technical details is also welcome.
> 
> 
> 
> 	WCS Client Driver for GDAL
> 	==========================
> 
> Purpose
> =======
> 
> To implement a new GDAL driver for accessing a remote WCS 
> coverage in a manner analygous to any local raster dataset.
> 
> 
> Technical Details
> =================
> 
> Overall
> -------
> 
> The WCS coverage driver will read a service description file 
> to establish the parameters it requires to make GetCoverage 
> calls on a WCS server.  If the the service description file 
> lacks DescribeCoverage information for the coverage, the 
> driver will invoke DescribeCoverage, and fill in the service 
> description file on disk.
> 

In terms of MapServer, this is a bit different than the other OWS client
approaches (i.e. the code is in MapServer itself for WMS, WFS client).
I would be interested in how this functionality will "roll up" to
MapServer and what the relationship is.  Just want to make sure there
are little/no duplications.

> Specification Features
> ----------------------
> 
> Version:
>   o The driver will support WCS version 1.0.0.
>   o The driver will support WCS version 1.1.0 based on 
> current draft. (Phase 2)

I'm pretty sure 1.1.0 implements the owsCommon specification.  It would
be valuable when implementing anything from owsCommon that it can be
reused in some way.  I could see this being leveraged in MapServer (even
as a code copy/paste) for the logic.

>   o The driver will *not* support WCS version 0.7.
> 
> Encodings:
>   o The driver will support GET keyword/value requests.
>   o The driver will *not* support POST XML requests.
> 
> Georeferencing:
>   o The driver will support coordinate systems defined only 
> in "EPSG:xyz",
>     "AUTO:xyz" or "OGC:xyz" formats, not GML CRS encodings.
>   o The driver will support georeferencing information for 
> grids only if
>     it is presented as a RectifiedGrid element in the 
> DescribeCoverage.
> 
> NullValues:
>   o The driver will support only one "singleValue" null 
> values.  It will
>     not support multiple singleValues, nor intervals.
> 
> SupportedFormats:
>   o The driver will support images returned in GeoTIFF, DTED and NITF
>     formats.  Other formats will also be supported if GDAL 
> drivers exist
>     (such as HDF-EOS if GDAL is configured with HDF-EOS 
> format support).
>     At this time GML coverage responses will *not* be supported.
>   o The driver will support multi-part mime bundles as a mechanism to
>     return multiple files (such as JPEG+world file).  (Phase 2)
> 
> Interpolation:
>   o The driver will default to use of nearest neighbour interpolation.
> 
> Time/Parameter Support:
>   o Supported via overrides in the Service Description File.
> 
> 
> Service Description File
> ------------------------
> 
> The service description file is an XML format file that can 
> hold all the information GDAL requires to access a WCS 
> coverage, as well as various optional items that can be used 
> to control particular behaviors.
> 
> It may include:
> 
>   1) The service URL - the base url to which DescribeCoverage and
>      GetCoverage requests is made.
> 
>   2) The result of the DescribeCoverage request for the 
> coverage.  This
>      includes the spatial extents, and resolution.
> 
>   3) Override values for various parameters, including interpolation,
>      and request format.
> 
>   4) The pixel data type for the coverage.
> 
>   5) Request values for time and other parameter dimensions.
> 
>   6) The coordinate system in Well Known Text format 
> (overrides coordinate
>      system information in the DescribeCoverage).
> 
>   7) The geotransform for the coverage (overrides values in the
>      DescribeCoverage).
> 
>   8) A preferred block size, and other reading strategy 
> information that
>      may affect performance.
> 

Frank: you may want to look at what we did for the OWSContext work for
binding to a WCS in a context doc.

> 
> Remote Access Strategies
> ------------------------
> 
> Getting respectable performance for client applications 
> accessing remote WCS servers will in large part depend on the 
> approach taken to turning local requests into remote 
> requests.  In particular, because some GDAL client 
> applications take a fine grained approach to requesting data 
> from drivers, it can be very helpful to aggregate several 
> requests based on some blocking scheme.  But other 
> applications may make single large read requests of exactly 
> all the data required.  It is desirable to handle these as a 
> single request to the remote WCS server that pull back just 
> the desired data.
> 
> Because the of diversity of situations several read 
> strategies will be supported.
> 
> 1) Block oriented with caching.  All requests to the remote 
> server will made based on the block size, and the blocks will 
> be cached normally in the GDAL memory block cache.  The 
> default block size should be substantial, perhaps 2MB blocks 
> the width of the image.
> 
> 2) 1:1 RasterIO() to GetCoverage. Each RasterIO() request 
> will result in a remote GetCoverage request at the resolution 
> and for the exact region requested.  Because of the irregular 
> nature of requests, caching is impractical.
> 
> 3) Hybrid.  Block oriented with caching will be used for 
> smallish requests, while very large RasterIO() requests will 
> be done as an immediate call but expanded to block boundaries 
> and the result pushed into the block cache.  The benefit is 
> that several blocks may be handled in a single request.
> 
> The default operation will be to use the Hybrid mode.  
> Specific strategies and block sizes can be forced via the 
> service description file.
> 
> Overviews
> ---------
> 
> WCS servers will be represented to the client application as 
> having power of 2 reduced resolution overview layers.  When 
> using block oriented IO, requests will be made at reduced 
> resolutions based on the overview levels when appropriate.
> 
> Other Issues
> ------------
> 
> 1) The DescribeCoverage information does not include any clue 
> as to the actual pixel data type of data.  In order for 
> GDALOpen() to return for a WCS based on a service description 
> file that contains only the url, it will be necessary to 
> perform a DescribeCoverage(), and also to fetch a small 
> sample file to establish the pixel data type.
> 
> 2) GDAL currently supports opening several file formats from 
> "in memory"
> buffers.  Other file formats do not support this.  As part of 
> this project, the driver will first attempt to open the 
> returned datasets using the in-memory mechanism.  If that 
> fails, the data will be written to a temporary file on disk 
> for access.  Suitable logic for placing temporary files will 
> be needed.
> 
> 3) The WCS client capability will need to be able to do 
> remote HTTP requests.
> I have used the curl open source library 
> (http://curl.haxx.se/libcurl) in several projects very 
> successfully, and would plan to use it for GDAL WCS as well.  
> It will be a requirement for building the WCS support.
> 
> 
> Testing
> =======
> 
> The WCS client will be tested against several WCS servers, 
> including hopefully UMN MapServer, IONIC Red Spider, 
> Cubewerx, and PCI Geomatics WCS servers.  The GDAL "autotest" 
> suite will also be extended with test scripts for the WCS 
> client driver operating against stable public WCS servers.
> 
> 
> Documentation
> =============
> 
> Standard "user" documentation will be written for the WCS 
> driver in HTML format.  It will include a detailed 
> specification for the service description file, and all it's 
> parameters.
> 
> 
> Intellectual Property
> =====================
> 
> The WCS driver will become a regular part of the GDAL 
> library, and it's source code and documentation will be 
> released under the normal GDAL open source license.
> 
> 
> Timetable
> =========
> 
> Phase 1 (essentially WCS 1.0.0) will be completed within 5 
> weeks of project approval.
> 
> Phase 2 (WCS 1.1.0) will be completed within 7 weeks of phase 
> 1 completion if approved.
> 
> Best regards,
> -- 
> ---------------------------------------+----------------------
> ----------
> ---------------------------------------+------
> I set the clouds in motion - turn up   | Frank Warmerdam, 
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | President OSGeo, 
> http://osgeo.org
> 
> _______________________________________________
> Gdal-dev mailing list
> Gdal-dev at lists.maptools.org
> http://lists.maptools.org/mailman/listinfo/gdal-dev
> 




More information about the Gdal-dev mailing list