[GRASS-dev] r.out.gdal, again

[Markus Metz]

Hi,

I have a wishlist for r.out.gdal together with a new version (link
below), and would like to put it up for discussion:
Changes in my version are:

Colortable export

A new flag -c is provided to export a colortable, default to no.
Colortables in GeoTIFF files are sometimes not properly rendered by
other GIS applications and the file might then be displayed all black,
even when the raster values are correct. The same GeoTIFF file without
colortable is displayed ok. Requested colortable export only works when
there is an entry in directories colr or colr2.

GRASS has no control over how a colortable is written to the exported
file, this is handled by GDAL and dependent on the file format. GRASS
uses intelligent color rules, whereas e.g. GeoTIFF needs a color table,
i.e. one entry for each potential, not actual data value.

Nodata value handling

If a nodata value was supplied, this value is tested whether it is
within the range of the selected data type and adjusted if necessary.
Specifying e.g. a nodata value of -9999 and using Byte as data type
(range 0 - 255) is no longer accepted, it will be adjusted to 0.
Similarly, a nodata value of 9999 would be adjusted to 255 for Byte
type. More generally, if a supplied nodata value is smaller than the
supported minimum, it is adjusted to the supported minimum, and if a
supplied nodata value is larger than the supported maximum, it is
adjusted to the supported maximum.

Current region extends aligned to raster resolution

The old r.out.gdal (GRASS-6.2.x) worked with the extends and resolution
of the raster to be exported. The new r.out.gdal C-module works with the
extends and resolution of the current region. My version works with the
extends of the current region and aligns them to the input raster map
resolution to avoid undesired implicit resampling if the resolution of
the current region does not match the input raster resolution. If a
raster map layer should be exported in a different resolution the raster
must now be resampled prior to exporting.

Raster band statistics

Raster band statistics (min, max, mean, stddev) are written to the
exported file if supported, GDAL decides. Since r.out.gdal uses current
region extends, these statistics are calculated for the current region
extends, replacing NULLs with the nodata value. I'm not sure if it would
be better to exclude NULLs for raster band statistics.

These statistics can be used by other GIS applications to get the actual
data range (an argument for inclusion of nodata values in raster stats),
but not many GIS applications actually make use of this information.
ESRI products e.g. initially report the potential data range, not the
real data range. This is usually not an issue with Byte type because of
its small range, but with e.g. Int32 or Float32, ESRI products often
initially display an all black image (not only GeoTIFF, also e.g. ESRI
.hdr labelled files). Adjusting the display mode (Symbology) helps.

Data type and range test

If the range of the raster to be exported is not covered by the selected
data type, r.out.gdal is aborted with an error message giving
information on the range of the raster to be exported as well as the
range of GDAL data types. The user can then easily compare the raster
data range with the ranges of GDAL data types and select an appropriate
data type.

Updated documentation

Documentation is updated, amongst other minor changes with ranges of
GDAL data types and recommended settings for best allround compatibility
of GeoTIFF files. The recommended settings are based on discussions in
GRASS mailing lists, GRASS ticket 73, GDAL documentation and own
testing. Apart from colortable export defaulting to no, these
recommended settings are not enforced or set as default.

Maciek looked into GeoTIFF optimisation, e.g. adding overviews, tiles
vs. strips, strip dimensions, internal compression methods. Since
GeoTIFF is so widely used, this information could also be included in
the r.out.gdal documentation.


This version of r.out.gdal was compiled with
grass-6.4.svn_src_snapshot_2008_08_16 and gdal-1.5.2.
Source code has been formatted using tools/grass-indent.sh

I tested with QGIS-0.9.1 on Linux 64bit, QGIS -0.11.0-1 on XP,
SAGA-2.0.3 on XP, gvSIG-1.1.1 (32bit binary) on Linux 64bit, gvSIG-1.1.1
on XP, PCI Geomatica FreeView 9.1 on XP, ER Viewer 7.2 (ER Mapper) on
XP, and ArcMap 9.2 (ESRI) on XP. It is not so easy to get hold of GIS
applications, also commercial, that do not use gdal...
Tests were performed with the elevation raster map layer in the North
Carolina sample dataset, creating a MASK for elevation values <= 70m,
needed for testing of nodata handling. The elevation raster was exported
as Byte, UInt16, Int32, and Float32, once with colortable (Byte and
UInt16 only, colortable export for other data types not supported by
gdal), once without colortable, always as GeoTIFF.

Test results:

QGIS-0.9.1 on Linux 64bit
display: all ok
colortable: all ok
cell values: all ok
nodata: all ok

QGIS-0.11.1 on XP
display: all ok
colortable: all ok
cell values: all ok
nodata: all ok

gvSIG-1.1.1 (Linux 32bit binary) on Linux 64bit, probably Java problems
display: all fail
colortable: all fail
cell values: all fail
nodata: all fail

gvSIG-1.1.1 on XP
display: all ok
colortable: fail for Uint16, ok for Byte
cell values: all ok
nodata: nodata not supported

PCI Geomatica FreeView 9.1 on XP
display: all ok
colortable: all ok
cell values: all ok
nodata: nodata not supported

ER Viewer (ER Mapper) on XP
display: fail for UInt16 with colortable, all other ok
colortable: fail for UInt16, ok for Byte
cell values: all ok
nodata: cell value query not supported

SAGA-2.0.3 on XP
display: all ok (colortables ignored)
colortable: all fail
cell values: all ok
nodata: all ok

ESRI ArcMap 9.2 on XP
display: all ok, but all without colortable and other than Byte need
manual adjustment which is painfully slow, even for these small raster
layers (Properties -> Symbology -> Stretch -> Minimum-Maximum)
colortable: all ok
cell values: all ok
nodata: all ok


Source code with README file is available at
http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz

The diff is larger than the source code, therefore I did not attach a diff.

Please excuse this long mail, but I wanted to explain my wishlist and
had to provide test results so I could show what output is readable where.

Regards,

Markus