[Gdal-dev] Nodata bitmasks / null processing

[Simon Perkins]

To throw in my two cents, I would prefer to see something like option 3. 
Option 2 just sounds horrible and inefficient (and I guess you don't 
like it). And if we're going to the trouble of adding an extra special 
alpha band for images with NODATA masks, then why not go just a little 
further and make it an explicit nodata mask?

Seems to me that this extra API could be tacked on fairly neatly to the 
existing NODATA mechanism, without that much disruption. Existing apps 
can continue to use the old NODATA mechanism, while newer apps can use 
this one, with the default implementation you suggest, which means that 
we don't need to modify any format drivers initially. I'm not sure I 
understand your concerns about how it might be complicated to implement 
for some formats with complicated blocking schemes - if you can write a 
RasterIO() method for these formats and you can specify a NODATA value, 
are there any new problems that weren't there before?

It is of course essential that the existing NODATA API continues to be 
supported.

Cheers,

Simon


Frank Warmerdam wrote:
> Folks,
>
> Currently GDAL supports two mechanisms for identifying nodata/transparent
> areas in a raster.  One is use of a particular pixel value as a nodata
> value (returned by the GDALRasterBand::GetNodataValue() method) and 
> the other
> is by specifying an alpha band with alpha=0 marking transparent areas.
>
> However, a number of file formats support a concept of "nodata masks" -
> essentially bitmasks marking areas of the raster that are nodata 
> areas.  But
> this is accomplished without having to reserve a special marker value 
> to be
> the nodata value.
>
> This is the case with GRASS for instance, and the GDAL GRASS driver 
> actually
> jumps through hoops to promote byte data to int16 and overlay a nodata 
> value.
> However, this is cumbersome and violates the underlying data model of the
> format.
>
> This issue also arises with other formats, including a new JPEG variation
> with a nodata bitmask appended to the file.  I also feel that nodata 
> masks
> can offer a better
>
> A client has requested support for nodata bitmasking be incorporated into
> GDAL in some appropriate fashion.  I can see a few approaches:
>
> 1) Pseudo-Alpha
>
> In this approach the nodata mask is treated as a sort of alpha band with
> values of 0 and 255.  One downside to this is that alpha bands are not
> clearly associated with a particular band, so if you wanted to have
> distinct nodata bitmasks for each band in a dataset there would be
> no explicit association between particular alpha bands and particular
> original bands.
>
> 2) Data Promotion and NODATA pixel value creation
>
> This is the approached used now in the GDAL GRASS driver.  A nodata value
> outside the current valid range is created within the driver, and if
> necessary the pixel data type is promoted to a larger type to make space
> for this "extra value" value.  For instance Byte pixel type would be
> promoted to UInt16 so we could use 256 as our nodata value.
>
> 3) Introduce Explicit Nodata Bitmap API
>
> Extend the GDAL API to include an option for reading a validity bitmask
> in the RasterIO() call - or perhaps a new BitmaskIO() method paralleling
> the RasterIO() method.
>
> Potentially the default implementation of this method could generate a
> bitmask based on the raster data and nodata pixel values if they are
> present so that applications could use this one method for both models.
>
> In one sense this is the cleanest solution, but it is also by far the
> most disruptive.  It requires new support in applications that want
> to take advantage of it, further complicates the GDAL API and worst, it
> may be very hard to implement properly for formats without bitmasks, but
> with complicated resampling or blocking logic.
>
> I am looking for some feedback from the community.  In particular I'm
> interested in examples of file formats that include bit masking.
>
> Best regards,