[GRASS-dev] r.out.gdal, again

Tue Aug 19 19:33:30 EDT 2008

On Tuesday 19 August 2008, Markus Metz wrote:
> Hi,
>
> I have a wishlist for r.out.gdal together with a new version (link
> below), and would like to put it up for discussion:
> Changes in my version are:
>
> Colortable export
>
> A new flag -c is provided to export a colortable, default to no.
> Colortables in GeoTIFF files are sometimes not properly rendered by
> other GIS applications and the file might then be displayed all black,
> even when the raster values are correct. The same GeoTIFF file without
> colortable is displayed ok. Requested colortable export only works when
> there is an entry in directories colr or colr2.
>
> GRASS has no control over how a colortable is written to the exported
> file, this is handled by GDAL and dependent on the file format. GRASS
> uses intelligent color rules, whereas e.g. GeoTIFF needs a color table,
> i.e. one entry for each potential, not actual data value.
>
> Nodata value handling
>
> If a nodata value was supplied, this value is tested whether it is
> within the range of the selected data type and adjusted if necessary.
> Specifying e.g. a nodata value of -9999 and using Byte as data type
> (range 0 - 255) is no longer accepted, it will be adjusted to 0.
> Similarly, a nodata value of 9999 would be adjusted to 255 for Byte
> type. More generally, if a supplied nodata value is smaller than the
> supported minimum, it is adjusted to the supported minimum, and if a
> supplied nodata value is larger than the supported maximum, it is
> adjusted to the supported maximum.
>
> Current region extends aligned to raster resolution
>
> The old r.out.gdal (GRASS-6.2.x) worked with the extends and resolution
> of the raster to be exported. The new r.out.gdal C-module works with the
> extends and resolution of the current region. My version works with the
> extends of the current region and aligns them to the input raster map
> resolution to avoid undesired implicit resampling if the resolution of
> the current region does not match the input raster resolution. If a
> raster map layer should be exported in a different resolution the raster
> must now be resampled prior to exporting.
>
> Raster band statistics
>
> Raster band statistics (min, max, mean, stddev) are written to the
> exported file if supported, GDAL decides. Since r.out.gdal uses current
> region extends, these statistics are calculated for the current region
> extends, replacing NULLs with the nodata value. I'm not sure if it would
> be better to exclude NULLs for raster band statistics.
>
> These statistics can be used by other GIS applications to get the actual
> data range (an argument for inclusion of nodata values in raster stats),
> but not many GIS applications actually make use of this information.
> ESRI products e.g. initially report the potential data range, not the
> real data range. This is usually not an issue with Byte type because of
> its small range, but with e.g. Int32 or Float32, ESRI products often
> initially display an all black image (not only GeoTIFF, also e.g. ESRI
> .hdr labelled files). Adjusting the display mode (Symbology) helps.
>
> Data type and range test
>
> If the range of the raster to be exported is not covered by the selected
> data type, r.out.gdal is aborted with an error message giving
> information on the range of the raster to be exported as well as the
> range of GDAL data types. The user can then easily compare the raster
> data range with the ranges of GDAL data types and select an appropriate
> data type.
>
> Updated documentation
>
> Documentation is updated, amongst other minor changes with ranges of
> GDAL data types and recommended settings for best allround compatibility
> of GeoTIFF files. The recommended settings are based on discussions in
> GRASS mailing lists, GRASS ticket 73, GDAL documentation and own
> testing. Apart from colortable export defaulting to no, these
> recommended settings are not enforced or set as default.
>
> Maciek looked into GeoTIFF optimisation, e.g. adding overviews, tiles
> vs. strips, strip dimensions, internal compression methods. Since
> GeoTIFF is so widely used, this information could also be included in
> the r.out.gdal documentation.
>
>
> This version of r.out.gdal was compiled with
> grass-6.4.svn_src_snapshot_2008_08_16 and gdal-1.5.2.
> Source code has been formatted using tools/grass-indent.sh
>
> I tested with QGIS-0.9.1 on Linux 64bit, QGIS -0.11.0-1 on XP,
> SAGA-2.0.3 on XP, gvSIG-1.1.1 (32bit binary) on Linux 64bit, gvSIG-1.1.1
> on XP, PCI Geomatica FreeView 9.1 on XP, ER Viewer 7.2 (ER Mapper) on
> XP, and ArcMap 9.2 (ESRI) on XP. It is not so easy to get hold of GIS
> applications, also commercial, that do not use gdal...
> Tests were performed with the elevation raster map layer in the North
> Carolina sample dataset, creating a MASK for elevation values <= 70m,
> needed for testing of nodata handling. The elevation raster was exported
> as Byte, UInt16, Int32, and Float32, once with colortable (Byte and
> UInt16 only, colortable export for other data types not supported by
> gdal), once without colortable, always as GeoTIFF.
>
> Test results:
>
> QGIS-0.9.1 on Linux 64bit
> display: all ok
> colortable: all ok
> cell values: all ok
> nodata: all ok
>
> QGIS-0.11.1 on XP
> display: all ok
> colortable: all ok
> cell values: all ok
> nodata: all ok
>
> gvSIG-1.1.1 (Linux 32bit binary) on Linux 64bit, probably Java problems
> display: all fail
> colortable: all fail
> cell values: all fail
> nodata: all fail
>
> gvSIG-1.1.1 on XP
> display: all ok
> colortable: fail for Uint16, ok for Byte
> cell values: all ok
> nodata: nodata not supported
>
> PCI Geomatica FreeView 9.1 on XP
> display: all ok
> colortable: all ok
> cell values: all ok
> nodata: nodata not supported
>
> ER Viewer (ER Mapper) on XP
> display: fail for UInt16 with colortable, all other ok
> colortable: fail for UInt16, ok for Byte
> cell values: all ok
> nodata: cell value query not supported
>
> SAGA-2.0.3 on XP
> display: all ok (colortables ignored)
> colortable: all fail
> cell values: all ok
> nodata: all ok
>
> ESRI ArcMap 9.2 on XP
> display: all ok, but all without colortable and other than Byte need
> manual adjustment which is painfully slow, even for these small raster
> layers (Properties -> Symbology -> Stretch -> Minimum-Maximum)
> colortable: all ok
> cell values: all ok
> nodata: all ok
>
>
> Source code with README file is available at
> http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz
>
> The diff is larger than the source code, therefore I did not attach a diff.
>
> Please excuse this long mail, but I wanted to explain my wishlist and
> had to provide test results so I could show what output is readable where.
>
> Regards,
>
> Markus
>
>

Nice work. I briefly tested this with some of the data in the spearfish 
location, and it seemed to work. Have you appended this information to the 
original ticket related to r.out.gdal not functioning correctly?

Great job, looking forward to a functioning r.out.gdal (finally!).

Cheers,

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341