[GRASS5] 0 != no data

[Roger Bivand]

Justin and friends,

Would this modified version answer Moritz' question? I agree with him that
we might risk getting a limited response because a regular 4.* user
mightn't grasp the issue. I haven't got a 4.3 installed, but as far as I
remember, there aren't any NULL values in the 4,* series are there? Can
someone please check? I've also added a third, compromise, choice - remove
0 as NULL from 5.1, but retain it in 5.0: the motivation is to lower the
threshold for 4.* users migrating to 5.*. I shortened the draft by taking
out the script details - they are valuable, but could be included as a
link to a note on the website.

Roger

------------------------------------------------------------
Subject: Representation of "no data": we need your opinion

Hello all users

Could we please ask you to review the issue of the representation of
"no data" described below and give us your views on the alternatives we
propose? This affects users of GRASS 4.* considering moving to GRASS 5.*,
and users of GRASS 5.0 beta who depend on using legacy data. This is your
chance to influence a major decision which may impact your use of GRASS!

Recently on the developers' mailing list, there has been a discussion
concerning the current representation of "no data" in raster map
layers. When working with categories/classes, it was convenient to use
integer zero (0) to express "no data", because the classes each had
non-zero numbers - like land-use classes. This is the position in GRASS
up to 4.*. GRASS 5.0 beta integrated experimental work done over many
years by people needing both floating-point raster values, and needing
to use numeric zero as a number.

This meant that representing "no data" raised a conflict of interest:
people with legacy data didn't want their "no data" (0) to "come back to
life", while the numerical people really needed zero, as in zero slope,
to have a meaning other than "no data". The term introduced to cover
"no data" is "NULL", which can include both "no data available", and
other situations where no numeric representation exists, like log of a
negative number, or dividing by zero.

There were then basically 2 groups of users in the mid 1990's:

1. Those who worked mostly with classes and had a large number of 
   files with 0 set as "no data"

2. Those who started to use raster maps as representation of
   continuous fields and 0 was a data value

The majority of the users were in group 1 and thus, the decision was
made to keep the feature that 0 could be interpreted as "no data".

In the discussion on the developers' list, some developers have indicated
that, since GRASS 5.0 implements the concept of NULL (which is represented
by machine dependent constants), it is confusing to also have the feature
that 0 can be interpreted as NULL, especially to new users of GRASS. NULL
is implemented as a separate mask of bits representing "data"/"no data",
in principle for each raster map layer, but not all modules generate them
automatically, meaning that users not knowing the history get mixed up.

In order to eliminate this confusion, a proposal was made that 0
should never be considered as "no data" in GRASS 5.0. Also, in order to
facilitate those users who wish to use their GRASS 4.x files with the
new GRASS 5.0, we would provide a script to assist them in converting
the 0 values of their data to appropriate NULL values in GRASS 5.0.

What we would like you to do is provide us with your opinion on whether
we should keep the feature of 0 being interpreted as "no data". To make
things a little easier, we would like you to reply to this post stating
which of the following options you wish to support:

    a) Full legacy interoperability: keep the feature of 0 being
       optionally interpreted as "no data"

    b) Transition from GRASS 5.0 onwards: keep the feature of 0 being
       optionally interpreted as "no data" in GRASS 5.0 only, eliminating
       it in subsequent releases
       
    c) Transition now: eliminate the feature from GRASS 5.0, but provide
       a script to convert GRASS 4.x files to GRASS 5.0 format for those
       who wish 0 to be NULL

Thank you for your time in this matter.

------------------------------------------------------------
-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand at nhh.no
and: Department of Geography and Regional Development, University of
Gdansk, al. Mar. J. Pilsudskiego 46, PL-81 378 Gdynia, Poland.