[gdal-dev] JPEG in GeoTIFF + Masks

[Even Rouault]

Hi Robert,

> Hi folks,
>
> After a bit of advice...
>
> JPEG compression is fast and pretty space-efficient for raster imagery. But
> it's RGB-only.

You could do 'gdal_translate rgba.tif rgba_jpeg.tif -co COMPRESS=JPEG' but
that's not what you want, since the alpha band would be JPEG compressed too,
leading to undesired artifacts when reading it.

> JPEG2000 supports RGBA, has built-in overviews, but it can be slow &
> CPU-hungry.
> TIFF with deflate/lzw supports RGBA but is big for imagery.
>
> For simple RGB originals where imagery is later reprojected, all the alpha
> band in the reprojected version ends up being is a boolean mask with a 0
> (nodata) or 255 (data) value.
>
> Ideally I want something that is JPEG compressed images, has an alpha band
> mask, and where I can create overviews.
>
> http://www.gdal.org/frmt_gtiff.html talks about internal nodata masks,
> which sounds relevant, but doesn't really explain how to create them or how
> they work? I'm guessing here, but there's a normal tiled JPEG image in the
> TIFF file, and a tiled mask band with 1bpp, which could presumably be
> compressed further with deflate/etc too.

Exactly that. In TIFF, internal mask bands are secondary 'directories' that are
tiled, 1bpp and deflate compressed.
See references to 'Transparency mask' in
http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf

To transform a RGBA into a RGB JPEG-in-TIFF+internal_mask_band

gdal_translate rgba.tif rgbmask.tif -b 1 -b 2 -b 3 -mask 4 -co TILED=YES -co
COMPRESSS=JPEG --config GDAL_TIFF_INTERNAL_MASK YES [-co PHOTOMETRIC=YCBCR]

If you don't specify  --config GDAL_TIFF_INTERNAL_MASK YES, a .msk file
according to http://trac.osgeo.org/gdal/wiki/rfc15_nodatabitmask will be
produced

> And at read-time they could come
> out as RGBA?

Hum, no. They will be seen as a per-dataset mask band, but not as a standalone
alpha band.

Actually the good habit would be to use GetMaskBand() systematically on a
dataset, since it can deal with
'real' alpha bands, dedicated mask bands, implicit mask bands when a NoData is
set, etc... (see
'Default GetMaskBand() / GetMaskFlags() Implementation' of above mentionned
RFC15)

>
> For mosaicing ... are they supported by overviews and VRTs in a sensible
> way? For VRTs - the internal nodata value (no data in this pixel) would
> need to match the VRT nodata (no tile in this area). Last time I tried
> creating overviews for JPEG-in-TIFF alpha/nodata was handled bizarrely, but
> that could be all fixed by now.

VRT currently takes the last source to impose its values over the sources that
it may overlap and that are mentionned before in the VRT description. Including
alpha or mask band. This is admitedly quite a surprising behaviour, that would
probably worth an enhancement.

But you can use gdalwarp to do mosaic with sources with mask band. It should
handle them similarly to source alpha bands.
Regarding output, gdalwarp currently only knows how to generate alpha band, not
per-dataset mask bands. But with above gdal_translate, you can do that as a
post-processing.

gdaladdo should also do a sensible use of source mask bands (since it actually
use GetMaskBand()), and generate overviews of mask bands (you probably need to
set again --config GDAL_TIFF_INTERNAL_MASK YES if your source has internal mask
band, not sure if it still needed)

>
> Is there a way to convert a masked TIFF to a vanilla RGBA TIFF image for
> previewing in eg. image editing software?

Sure ! gdal_translate rgbmask.tif rgba.tif -b 1 -b 2 -b 3 -b mask

>
> Talking more about JPEG-in-TIFF, the docs say:
>
> > When translating a RGB image to JPEG-In-TIFF, using PHOTOMETRIC=YCBCR can
> make the size of the image typically 2 to 3 times smaller than the default
> photometric value (RGB). When using PHOTOMETRIC=YCBCR, the INTERLEAVE
> option must be kept to its default value (PIXEL), otherwise libtiff will
> fail to compress the data.
>
> Does interleave=pixel affect read performance cf =band for a RGB JPEG? It's
> within a tiled tiff and JPEG compression is both 3-band & tiled, so I'm not
> really sure where the interleaving fits in?

INTERLEAVE=PIXEL means that structure "R1G1B1,R2G2B2,etc..." (packed as strips
or tiles depending on layout selected with TILED)
INTERLEAVE=BAND means that structure "R1,R2,R3.....", later in the file
"G1,G2,G3,....", later in the file "B1,B2,B3......"

With JPEG compression, that's similar. The result is of course not the RGB
values, but their equivalent after DCT transform, quantization and Huffman
compression. But the layout is similar.

For YCbCr, as there is a subsampling of Cb and Cr components, you cannot
(reasonably) have INTERLEAVE=BAND. The components must be passed together to
libjpeg.

INTERLEAVE=PIXEL is actually the default used by GDAL when creating GTiff. It is
a reasonable one since when you have RGB imagery, you generally need to access
the 3 components of a pixel.

INTERLEAVE=BAND is more suite for scientific data, with many bands, where
accessing only once makes sense depending on the analysis you want to do.

Regards,

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com