[gdal-dev] JPEG compressed GeoTIFF ignores Nodata

[Even Rouault]

Le samedi 29 janvier 2011 20:28:02, Marius Jigmond a écrit :
> Thanks Even. I ran some more tests but that didn't quite work for me. I
> still get those noisy pixels after gdal_translate (in QGIS I set value 0
> for transparency and those pixels stand out). However, I noticed the
> following:
> 
> * nearblack -setmask shrinks the significant pixels area (expected?)
> * the near black noise pixels seem to follow a stair step pattern
> similar to the edge of the significant area but at a different scale
> (larger steps, if what I just said makes sense) and extend past the
> significant data areas in both masked and unmasked datasets.

Ah yes, nearblack has some default parameters that can account for this. See 
the -near and -nb parameters of http://gdal.org/nearblack.html

You can set them to 0 to have strict nearblack behaviour.

> > 
> I was aware of the fact that nodata values are band specific but I had a
> little brain glitch from working with gdaladdo (which honors nodata only
> as a triplet nodata). That's how I actually ran into the problem. I am
> trying to compress a RGB GeoTIFF using JPEG and then generate compressed
> overviews. We plan to serve some imagery through Geoserver and would
> like to stick to open-source solutions. The compressed datasets are
> significantly smaller and with overviews they are extremely responsive,
> rendering very fast). OpenJPEG v2 looks great but I haven't found
> support for it in Geoserver, yet.

To limit the issues with artifacts, I would compute the overviews on the 
uncompressed dataset and then translate the whole full resolution + overviews 
into a JPEG compressed TIFF with the new COPY_SRC_OVERVIEWS option of the GTiff 
driver.

I'm not sure that JPEG2000 would solve the issue. This would need to be 
checked if alpha channel is strictly preserved.

For the record, MapServer is able to use nodata mask band as transparency 
channel.

> 
> -marius
> 
> On Sat, 2011-01-29 at 16:13 +0100, Even Rouault wrote:
> > Le samedi 29 janvier 2011 15:47:43, Marius Jigmond a écrit :
> > > Hi Everyone,
> > > 
> > > I am having some trouble figuring out why adding JPEG compression to a
> > > RGB GeoTIFF results in Nodata pixel triplets becoming data pixel
> > > triplets. Is this expected behavior due to the noisy nature of JPEG
> > > compression or to the fact that these are border Nodata triplets? I
> > > know certain algorithms process the boundary between data and Nodata
> > > differently.
> > 
> > Marius,
> > 
> > Yes you did find the cause : the pixels maching nodata values will be
> > JPEG compressed, so you have no guarantee that the 0 value is preserved
> > through compression/decompression. I'd also note that the nodata concept
> > in GDAL is a per-band value (so it should not be interpreted as a nodata
> > "triplet")
> > 
> > A possibility to workaround this would be to create a "nodata mask band"
> > (see http://trac.osgeo.org/gdal/wiki/rfc15_nodatabitmask), but I'm
> > pretty sure that QGIS ignores those mask bands.
> > 
> > Anyway, if you want to try and create one, you can :
> > 
> > 1) Use nearblack to create a nodata mask band based on (0,0,0) triplet
> > nearblack -setmask -of GTiff -o out_with_mask.tif in.tif
> > 
> > (the -setmask option is new in GDAL 1.8.0)
> > 
> > 2) Then compress the dataset with
> > gdal_translate -co COMPRESS=JPEG -co TILED=YES -co JPEG_QUALITY=90 -co
> > PHOTOMETRIC=YCBCR out_with_mask.tif out_compressed.tif
> > 
> > Only the RGB channels will be JPEG compressed : the mask band will use
> > lossless compression so you won't have artifacts at the borders between
> > significant areas and nodata areas.
> > 
> > (You can add --config GDAL_TIFF_INTERNAL_MASK YES to make the nodata mask
> > internal to the GTiff file, instead of having a .tif.msk external file,
> > which will not make any difference for software using GDAL API but can
> > be more convenient to have a sinle file)
> > 
> > FYI, you can later reconvert the dataset into an uncompressed RGBA
> > dataset :
> > 
> > gdal_translate out_compressed.tif out_rgba.tif -b 1 -b 2 -b 3 -b mask
> > 
> > Best regards,
> > 
> > Even