[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not
work
GRASS GIS
trac at osgeo.org
Sat Oct 25 13:24:09 EDT 2008
#73: r.out.gdal tiff output does not work
--------------------------+-------------------------------------------------
Reporter: helena | Owner: grass-dev at lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords: r.out.gdal, tiff
Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by mmetz):
Replying to [comment:18 hamish]:
> Replying to [comment:11 mmetz] (and others):
>
Several of my previous descriptions about the conservative version of
r.out.gdal are out of date, e.g. the way statistics are calculated. I
would prefer not to stick strictly to my version, instead use it as ideas
for discussion.
>
> Spearfish's elevation.dted would be a better sample dataset to use here,
it is CELL 0-255. or convert spot.image into RGB bands.
> Other test datasets are the spearfish imagery60 data, and the North
Carolina dataset lsat7 images.
>
Using sample datasets available to all is a very good idea.
>
> > Still, datatypes other than Byte can only be read with spatial data
viewers that could make use of this colour table, but not all of them are
actually able to do so.
>
> Important to know when this is happening, not really our problem, and
not much we can do about it as long as we are writing metadata to spec.
>
I agree, GRASS can not fix or accommodate other viewers/packages.
>
> > It would help a lot to have a list with these data types and their
ranges in the man page so that the appropriate data type for file export
can be chosen.
>
> Yes. Can someone in the know supply some suggested text?
>
See the updated documentation in my version.
>
> > Maybe the color table is not so important, different people have
different preferences
>
> The color table will be carefully constructed/chosen by some people and
they may wish to keep it. e.g. r.colors.stddev addon script, r.cpt2grass
addon script, and NASA color tables given for MODIS sat data (see grass's
MODIS wiki page).
> These data are real (calibrated into sea surface temperature or mg/mL
chlorophyll) and also have specially designed color tables (incl log-
scale) which are nice to keep.
>
True, but if you preprocess such data for someone else who will be running
his/her own analyses, a colortable is not important. Sometimes it is
desirable to keep the colortable, sometimes not, therefore I would like to
have the option to keep it or not.
>
> > Suggested data visualization with an embedded color table is nice to
have, but coloring schemes are a matter of taste
>
> geotiff has a method for assigning color palettes, all I suggest is
populating that correctly, apparently we are currently (or were) failing
at.
>
I disagree, the colortable is populated correctly with the current svn
version (at least in my tests), but other GIS software may not be able to
make use of it. Nothing GRASS can do about that.
>
> > There is still the issue of nodata: a nodata value must fall into the
range of the chosen datatype, otherwise this information gets lost.
r.out.gdal does not check if a supplied nodata value is valid.
>
> Hopefully better in SVN now. If the user manually specifies an out of
range value, I don't mind respecting that.
>
Users may accidentally specify an out of range nodata value. That has a
similar effect like replacing NULL values in the GRASS raster. I would at
least like to show a warning.
>
> > For UINT16, the color table in the tiff file will always have 65535
entries, gdal does that.
>
> perhaps this should be addressed in GDAL then; only write palette
entries which are given.
>
File an enhancement for GDAL?
>
> > Have an option in r.out.gdal to choose whether a color table should be
written to the tiff file?
>
> For Uint16, how to just get the "COLOR_TABLE_RULE_RGB_0" part of this
without the "(RGB with 66535 entries)" part?
>
Done in my version.
>
> > basic statistics (min, max, mean, stddev) are calculated for the
current region, replacing NULL cells with nodata value (default or
specified) and then exported with GDALSetRasterStatistics()
>
> how is this better than "gdalinfo -stats" ?
I thought it could be better because other software could read the
metadata. But they don't, at least the ones I tested. So there is no
advantage in giving stats in the metadata.
>
> I don't understand, you convert NULL to nodata value in stats? So for a
CELL map with lots of NULL areas, if I make nodata=100 (well above the
number of used palette entries) or leave it at 255, the stats will be
skewed towards 100 or 255 instead of just considering the real data? huh?
"gdalinfo -stats" seems to do the right thing here.
>
Right,I changed that and ignore nodata, stats as calculated by my version
are now identical to stats calculated by gdalinfo.
>
> > colortable is only exported on request (new flag needs to be set) and
only if there is an entry in directories colr or colr2
>
> This is wrong, libgis is in charge of handling colors. Use the API not
raw file access. For one thing, otherwise you miss out on the default
mapping if there isn't a colr/ file.
>
G_read_colors() returns 0 if no entry in colr or colr2 is present, if an
entry is present it returns 1. That's what I used, not raw file access.
>
> > Additionally, a warning would be useful if the selected datatype does
not cover the range of raster values to be exported
>
> i.e. if the map was CELL 0-744 as in my elev_0 example earlier in this
ticket, but the user asked for Byte (0-255). I think that's a very good
idea.
>
> Start by moving range maxes into header #defines for each type from the
currently scattered "nodataval = -1E37f;" etc lines in the r.out.gdal
source code.
>
Also do that for range min's?
>
> If other software ignore valid GeoTIFF metadata tags, then that's their
problem and their responsibility to provide an alternative for their
users. All we can do is correctly implement the spec. If other software
read but incorrectly handle valid GeoTIFF metadata, then that's really
their bug to deal with.
>
Again, at least have the option. You can still tell people using such
software that they should try GRASS...
>
OK, the general question is, should GRASS consider that some packages
don't fully support all GeoTIFF or other file type features or not and
then strictly stick to the specs?
>
> > Apparently COLOR_TABLE_RULE entries are custom metadata not read by
other software.
>
> Any confirmation on this? It is custom to GRASS or custom to OSGeo
family (GDAL dependents) or suggested by the geotiff spec or ...???
>
My personal interpretation of the source code of r.out.gdal, needs
confirmation. I can only say that the tested packages apart from QGIS
don't make use of COLOR_TABLE_RULE_RGB_%d entries.
>
> > current region restored on exit
>
> module should not have to do that, it should set WIND_OVERRIDE variable
or so, so if module is interrupted by ctrl-C or whatever the temporary
change does not become permanent. (but as mentioned earlier, this violates
grass's raster module conventions and the feature should not be applied in
SVN)
>
Yes, it violates GRASS raster conventions, but in this case I can't see
why it would be desirable to change the resolution on the fly with
g.region, i.e. nearest neighbour for raster export. At least it should be
stated clearly in the documentation that r.out.gdal respects the current
region extends and resolution.
>
> > if supported, GDAL decides
>
> what does that mean? when is min/max etc not supported??
>
I don't no the specs of all raster types supported by GDAL. r.out.gdal can
try to export all metadata available, but some other formats support more
metadata, some less. It should not be of interest for GRASS, because GDAL
decides whether certain metadata are supported or not.
>
> > Tests were performed with the elevation raster map layer (range within
Byte, FCELL map) in the North Carolina sample dataset, creating a MASK for
elevation values <= 70m, needed for testing of nodata handling with
nodata=0. The elevation raster was exported as Byte, UInt16, Int32, and
Float32, once with colortable (Byte and UInt16 only, colortable export for
other data types not supported by gdal), once without colortable, always
as GeoTIFF.
>
> Please supply map names and exact commands so we can replicate.
>
I will do so in a separate reply, this one is long enough already.
>
> > QGIS-0.9.1 on Linux 64bit
> > QGIS-0.11.1 on XP
> > ...
> > ESRI ArcMap 9.2 on XP
>
> this is handy information for users to be aware of, but (again) not much
we can do about 3rd party support beyond coding to geotiff
spec/recomendations.
>
In the end, it boils down to colortables in UInt16 files, therefore my
request for an option to export colortables. That doesn't harm and can do
good.
>
> > I would prefer to use r.out.gmt to get the GRASS color rules into an
extra file (GMT .cpt). This would also be a workaround for export file
types other than Byte and UInt16 where GDAL does not support writing out a
colortable. But then I don't know how widely GMT .cpt files are supported.
>
> Since it has been some months since the original bug report, and the
code has changed, could we please restate what this bug actually is
(rather than fuzzy improved r.out.gdal niceness), and some spearfish or NC
sample-dataset instructions for recreating it? After studying the ticket
from beginning to end I can't say where the GRASS bug is, beyond:
>
> * provide warning if map data overflows requested data type=
> * type limits to #defines in the code, and human readable to the help
page
> * how to make GDAL stop writing color rules for UInt16 once Maximum= is
reached (instead of writing out 66535-Max empty rules)
Ask Frank Warmerdam if this is necessary? This is GDAL code, AFAIK the
GDAL API does not provide such an option.
>
> what else?
>
* an option for colortable export:-)
* Nodata handling: more documentation, warning or error messages if out
of range because this would be implicit raster editing.
>
> And we must remember that r.out.gdal is not just for GeoTIFF files..
>
... but for many many other formats. Too many for testing? Leave as much
error handling as possible to GDAL?
In the end, r.out.gdal as it is now may not be as bad as the length of
this ticket suggests, and the 3 points of Hamish plus my two points would
not require a lot of rewriting.
Regards,
Markus
--
Ticket URL: <https://trac.osgeo.org/grass/ticket/73#comment:19>
GRASS GIS <http://grass.osgeo.org>
More information about the grass-dev
mailing list