[gdal-dev] Possible bug in gdaldem color-relief, zero always transparent

[Even Rouault]

Le Friday 26 February 2010 00:13:22 Matt Williamson, vous avez écrit :
> Ok, got some clarification.
>
> The _real_ problem was that my dataset did not have it's NODATA value set
> properly. That being the case, gdaldem apparently assumes 0 is the NODATA
> value...sort of...if you have an "nv" line in the color table, perhaps.
> Since I had a "nv" line in my color table, it seems that the setting for 0
> and the setting for nv were conflicting. The result was still rather odd,
> so I might still toss this in the bug tracker and see what you guys think
> of the behavior.

Yes, please do. While reviewing the code, it appears that in the case 'nv' is 
defined but there's no NODATA value defined for the dataset, it is wrongly 
interprated as 0. I agree it should be ignored in that case. So you ended up 
with 2 entries in the color file with the same key, which leaks to an 
undefined behaviour for which entry will be used.

>
> I tried removing the nv entry from the color table, and the results were
> perfectly sensible. Removing the explicit zero entry and leaving the nv
> setting produced strange results...it seems that in this case, the value
> for zero gets set to black with the opacity of the nv entry, unless the nv
> entry has opacity 255, in which case the zero value is the color specified
> for nv.
>
> It's probably one of those cases where if you have a nv entry specified for
> a dataset without a NODATA value, then the behavior is undefined...which is
> fine, but it took me a while to figure out what the heck was going on.
> What's really strange to me though is that the actual NODATA pixels
> (-30000, in this case) also seemed to be taking on the nv color settings,
> correctly, even though no NODATA value was set...a fluke probably, but
> whatever.
>
> On a separate note, the reason I had a file with no NODATA value specified
> was that I expected gdalwarp to pass-through the NODATA value from the
> source file if none of the -*nodata switches were present--but it doesn't.
> I can mostly understand this, but it's somewhat counterintuitive since
> gdal_translate works this way.

-srcnodata should not been needed. If not specified, gdalwarp will use the 
nodata value of the source bands.
But currently, you need to specify -dstnodata. I guess the reason is 
historical / backward compatibility with behaviour of previous versions of 
gdalwarp.

> Also, I am forced to specify a value for 
> -dstnodata to get it to work--but as it happens I'm trying to process
> multiple files in batch, which have different output data types, and
> different source NODATA values...is there a way to have gdalwarp
> passthrough the NODATA value from the source to the destination file
> automatically?

If you're on Unix/Linux, use of usual command line utilities should do it :

NODATA_VAL=$(gdalinfo in.tif | grep "NoData Value" | awk '{print $2}' | 
awk -F '=' '{print $2}')
if "$NODATA_VAL" != ""; then
	gdalwarp -dstnodata $NODATA_VAL in.tif out.tif
else
	gdalwarp in.tif out.tif
fi

But this could be worth an enhancement ticket. I'd imagine something 
like "gdalwarp -dstnodata source src.tif dst.tif" to use the source nodata 
values as dest nodata values.

>
> Thanks again,
>
> -Matt
>
> On Feb 23, 2010, at 3:55 PM, Even Rouault wrote:
> > Hum, then I have no clue. Yes I tested with a dataset with zeros.
> >
> > You can file a ticket on Trac with the dataset and the .clr file
> > attached, provided that the dataset is not too big (no more than 1 MB).
> > Otherwise, you'll have to provide a link to the dataset.
> >
> > Le Tuesday 23 February 2010 22:26:44 Matt Williamson, vous avez écrit :
> >> Hmm! Thanks, that is definitely not something I would have
> >> checked--However, I don't think it's my problem. I checked my .clr file
> >> I'm running, and I don't see anything out of the ordinary:
> >>
> >> $ xxd T_SFC.clr
> >> 0000000: 2d34 302e 3020 2031 3237 2032 3535 2032  -40.0  127 255 2
> >> 0000010: 3535 0a2d 3330 2e31 2020 3139 3220 3235  55.-30.1  192 25
> >> ...
> >> 0000070: 320a 2d31 302e 3020 2020 3634 2020 3634  2.-10.0   64  64
> >> 0000080: 2020 3634 0a2d 302e 3120 2020 3132 3720    64.-0.1   127
> >> 0000090: 3132 3720 3132 3720 3235 350a 3020 2020  127 127 255.0
> >> 00000a0: 2020 2020 3634 2020 3634 2031 3237 2032      64  64 127 2
> >> 00000b0: 3535 0a30 2e31 2020 2020 2036 3420 2036  55.0.1     64  6
> >> 00000c0: 3420 3132 3720 3235 350a 392e 3920 2020  4 127 255.9.9
> >> ...
> >>
> >> You ran it on a dataset with zeroes? Everything works properly for me
> >> except the zero values in the source GeoTIFF--everything else is colored
> >> as expected, with the correct alpha values.
> >>
> >> In fact (I probably should have mentioned this), when I first noticed
> >> the problem, I did not have the 0.1 value line in there, and it was
> >> actually interpolating the alpha from 0-255 between the values 0 and 9.9
> >> (i.e. it got progressively more opaque between those values).
> >>
> >> Also, FWIW, I encountered this issue under 1.7.0 and upgraded to 1.7.1
> >> to make sure it wasn't a bug fixed in .1, but no, it does it in both
> >> versions.
> >>
> >> -Matt
> >>
> >> On Feb 23, 2010, at 1:50 PM, Even Rouault wrote:
> >>> I think the problem lies in you .clr file. I pasted the content from
> >>> the email into a file, and when looking with an hexadecimal editor, I
> >>> see that there are several instances of a strange hexadecimal sequence
> >>> "0xc2 0xa0" that is used as a separator between the figures. If your
> >>> original .clr file has really those, and they are not an artifact of
> >>> pasting into the email, then this is the cause of your issue, as the
> >>> parser only supports spaces, tabulations, commas or colons. When fixing
> >>> the clr file, I get expected results.
> >>>
> >>> Le Tuesday 23 February 2010 20:12:14 Matt Williamson, vous avez écrit :
> >>>> Hello List,
> >>>>
> >>>> I may have encountered a bug in gdaldem color-relief. When I use it to
> >>>> apply a color map to a GeoTIFF band that has positive and negative and
> >>>> zero values, the zero pixels are always translated as having alpha 0,
> >>>> even if the color map file explicitly states zero values should be
> >>>> opaque.
> >>>>
> >>>> e.g., I'm using a command like this one:
> >>>>
> >>>> gdaldem color-relief MinT_SFC.EPSG_900913.tif colortables/T_SFC.clr
> >>>> pngs/test.png -alpha -b 1 -of PNG
> >>>>
> >>>> and a segment of the color table file reads as follows:
> >>>>
> >>>> -10.0   64  64  64
> >>>> -0.1   127 127 127 255
> >>>> 0       64  64 127 255
> >>>> 0.1     64  64 127 255
> >>>> 9.9    127 127 192
> >>>>
> >>>> I can provide a sample dataset that produces this problem, but I'm not
> >>>> sure what the etiquette on attaching files in this list is.
> >>>>
> >>>> Can anyone confirm they see the same behavior? I'm using 1.7.1,
> >>>> compiled locally. The behavior appears whether my -of is PNG or GTiff,
> >>>> so I don't think it's a driver issue.
> >>>>
> >>>> I know this has nothing to do with generating elevation maps, but it's
> >>>> too convenient a way to apply a color map to float data--I had been
> >>>> using mapserver just to make colorized PNGs, and gdaldem is so much
> >>>> cleaner.
> >>>>
> >>>> Thanks!
> >>>>
> >>>> -Matt
> >>>>
> >>>> _______________________________________________
> >>>> gdal-dev mailing list
> >>>> gdal-dev at lists.osgeo.org
> >>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev