[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work

[Markus Metz]

Glynn Clements wrote:
> Markus Metz wrote:
>   
>> I guess a minimum requirement would be that 
>> something exported with r.out.gdal and then imported again with 
>> r.in.gdal should be identical (taking into consideration the region 
>> settings during export) and not have any data loss, be it due to 
>> clipping data or due to nodata problems.
>>     
>
> That's only possible if the exported data is in a format which can
> hold the full range of the data. A UInt32 TIFF can't hold the full
> range of CELL data (it can't hold any negative values).
>   
A UInt32 can hold the full range of a CELL raster map if there are no 
negative values. What you mean is the potential range, what I mean is 
the data range actually present in the raster. Which still means that 
UInt32 doesn't really make sense as GDAL export type for GRASS rasters.
>> That would also mean that r.out.gdal with type = GDT_Int32 and nodata = 
>> 2147483648, NULL cells become -2147483648 but the nodata value in the 
>> metadata stays 2147483648 (gdalinfo on the export output), which in turn 
>> means that other software, also QGIS, does not see any nodata cells in 
>> the export output ->information loss about NULL cells. That's why I 
>> wouldn't use 0x80000000 as default nodata value for GDT_Int32 in 
>> r.out.gdal even if r.in.gdal has no problems with it.
>>     
>
> You're misinterpretating my use of "0x80000000" here. I'm talking
> about the C interpretation, which is the (signed) integer value
> -2147483648.
>   
OK, but I have tested r.out.gdal with type = GDT_Int32 and nodata = 
0x80000000. Same result: NULL cells become -2147483648 but the nodata 
value in the metadata says 2147483648 (gdalinfo on the export output), 
and other software including QGIS doesn't see any nodata cells. I 
haven't changed the part of the code of r.out.gdal that writes the 
nodata value to metadata, also not the code that writes the actual 
raster band.
>   
>> For r.out.gdal, there was discussion about not to override user options 
>> and instead issue an error or a warning (going for error now) that the 
>> nodata value is out of range.
>>     
>
> This isn't about overriding user options, but interpreting them
> correctly.
>   
Close to "the module knows better" which is not desirable according to 
Dylan and Moritz.
> It's debatable whether just using atof(nodataopt->answer) directly is
> actually a correct interpretation when the input map is CELL, or
> whether the code should use e.g. atoi(nodataopt->answer) or
> (CELL)atof(nodataopt->answer) for CELL inputs. Reading as int (or
> casting to it) will interpret 0x80000000 as -2^31, while reading as FP
> will interpret it as 2^31.
>   
If r.out.gdal should support out of range nodata values, and the nodata 
value is read with atof(), 2^31 should stay 2^31and not be cast to 
-2^31. The GUI description of the nodata option says that it is of type 
float (actually it's a double), i.e. suggesting that any value that is 
representable as floating point is ok.
> OTOH, reading as FP allows you to specify +2^31 as the nodata value
> when exporting CELL as UInt32. Maybe it would be better to read as
> double then cast to the destination type.
>   
Hmm, then it would be no longer possible to have an out-of-range nodata 
value in the metadata. I would support that solution, but it was 
explicitly requested to obey all user options when the -f flag is used, 
also writing the nodata value as it is to the metadata. It is then up to 
the user to figure out what happened to the NULL cells, assuming that 
the user knows what happens when using the -f flag.
>   
>> Currently, all default nodata values are 
>> within the range of the selected GDAL data type. User-defined nodata 
>> values are tested whether they are within the range of the selected data 
>> type, if not, r.out.gdal in trunk and devbr6 exits with an error 
>> message. r.out.gdal also exits with an error message if the nodata value 
>> is present in the raster to be exported in order to avoid data loss.
>> This is a rather strict and conservative approach because of the 
>> reported problems with nodata in r.out.gdal exports.
>>     
>
> Right.
>
> Even so, 0x80000000 is still the ideal value for exporting CELL data
> to either UInt32 or Int32. You just have to pick the correct
> interpretation of it (unsigned int or signed int respectively).
>   
The current as well as previous policy of r.out.gdal is to not to 
interpret the nodata value. It is written as is (currently double) to 
metadata, same like e.g. gdal_translate -a_nodata behaves.

Afraid of rewriting large parts of r.out.gdal: can't we just use 
-2147483648 as default nodata value for Int32 and 0 as default nodata 
value for UInt32? Getting tired of this UInt32 example, but again: 
(unsigned int) 0x80000000 is 2^31, yes? Then this value would be right 
in the middle of the range of UInt32, and can cause data loss when 
exporting a FCELL raster as UInt32 . For the sake of the argument, let's 
assume this FCELL raster has only positive values, some of them > 2^31, 
and any errors when casting from FCELL to UInt32 are acknowledged and 
accepted.

Considering that we agree that UInt32 is not really useful for exporting 
GRASS rasters, this discussion becomes very academic.

IMHO, with the new warnings/errors it is possible (not to say easy) to 
create r.out.gdal output that can be read and displayed by other 
software, all within the limits of that other software, and I would 
therefore not be willing to rewrite major parts of r.out.gdal to have 
more sophisticated interpretations of user options. I tried my best to 
address the issues mentioned in this ticket; for these nodata type 
casting issues someone else than me would be better qualified.

Markus M