Proposed change to GDAL FITS driver

[Simon Perkins]

Hi,

I'd like to make a tiny but just possibly incompatible change to
the FITS driver, so I'm writing this first to see if the change
is likely to affect anyone out there. If you don't use the FITS
driver in your code then please ignore this message. Even if you
do use the FITS driver, you can most likely ignore this message
anyway!

FWIW, the change I'm proposing only involves a single, completely
undocumented aspect of the driver, so I'm predicting that nobody
outside of the organization I work in (where the driver was
written) will be affected.

But if by some thosuand to one chance, what I'm suggesting is
actually going to affect you, then let me know ASAP!


Anyway...

The GDAL FITS driver is built on top of the cfitsio library. One
feature that this library supports is the ability to
automatically rescale data values as they are being read and
written from/to the image, through the use of the BZERO and
BSCALE keywords. This is most usually used to store floating
point data as a bytes. One simply sets the BSCALE and BZERO
keywords in the FITS file appropriately, and then data that is
written is automatically transformed to a value:

stored_value = (original_value - BZERO) / BSCALE

By choosing BSCALE and BZERO to map the original floating point
range onto 0 to 255, we can achieve a simple form of (typically)
4:1 compression. When reading values out of the image, the
inverse transformation is automatically performed.

Since the beginning, the GDAL FITS driver create() method has had
a couple of undocumented options: if you specify the options
BZERO and BSCALE with appropriate values in the creation options,
then you will end up with a file that looks like a floating point
file as far as GDAL is concerned, but actually stores the
underlying data as bytes using the specified scaling. Similarly,
if you open a byte FITS file that has the BZERO and BSCALE
keywords set, GDAL will tell you that the file is of floating
point type, and will silently transform the stored byte values
back into rescaled floating point values when you read them.

While this approach is convenient for the most common uses of
BSCALE and BZERO, it has a couple of problems:

- It only allows the user to map floating point types onto bytes.
In fact though the BSCALE/BZERO mechanism should allow any
combination of types, e.g. storing doubles as shorts, or ints as
bytes, using appropriate rescalings.

- It is sometimes convenient to set BSCALE and BZERO, but then to
do the rescaling yourself, i.e to write raw bytes or read raw
bytes even though BSCALE / BZERO is set.


To address these problems I'm proposing the following:

- Eliminate the sneaky remapping of data types that the FITS
driver currently performs when BSCALE/BZERO is set. It just
complicates things and is unnecessary. Worse it restricts the
functionality of the driver.

- Add a third option to the create() function: NOSCALE=YES. This
will disable the default scaling of data that is performed if the
BSCALE and BZERO options are specified, and allow the user to
write raw bytes if they want. Note that since the GDAL open()
method doesn't allow options (Frank: is this something that could
be usefully added to GDAL? A final optional argument to the C++
GDALOpen() method?), it's not possible to override the default
behavior of rescaling data read from a FITS file that has BZERO
and BSCALE set.



If you've never used the BSCALE/BZERO creation options then
you'll see almost no change. The only difference will be that if
you read from a FITS file that has the BSCALE and BZERO keywords
set, and you call GDALGetRasterDataType(), you will now see the
true underlying type of the image rather than a fake one. Reading
FITS data will happen exactly as before, with data being
automatically scaled using BZERO and BSCALE, if set, as it is
being read.


If you are using the BSCALE and BZERO creation options, and you
want to, say, store floating point data as scaled bytes, then
you'll now have to create the GDAL file as a byte file, rather
than a float file. The BSCALE and BZERO creation options will be
exactly as before - only the type argument will change.


Advantages of the new scheme are:

- You can use BSCALE and BZERO to store images as any type you
like - you're no longer restricted to bytes.

- The GDALGetRasterDataType() method doesn't lie anymore.

- You can optionally disable the automatic rescaling of data when
writing files.



If I don't hear any objections, I'll be submitting changes to CVS
in the near future, and hopefully the change will eventually
propagate through to a release version of GDAL, sometime in the
not too distant future.


Cheers,

Sy

-- 
Simon Perkins
Los Alamos National Laboratory, USA
http://nis-www.lanl.gov/~simes