[GRASS5] Presence of NULL data in raster maps?

[Glynn Clements]

Markus Neteler wrote:

> > In any case, we should arguably be getting rid of the null file and
> > just storing null values in the cell file. ISTR that the use of a
> > separate null bitmap was identified as a significant performance hit.
> 
> This was proposed a long time ago, I can't remember why the change
> wasn't done. Maybe the person lost track on the idea.

I suspect that nobody is confident that they fully understand the
implications of doing so.

> > > A bit dirty, but otherwise the
> > > cellhd file needed an addition (null: true or whatever).
> > 
> > Or you just accept the cost of scanning for nulls, or you assume that
> > all maps conceptually have null cells, even if, for some maps, the
> > number of null cells happens to be zero.
> 
> Scanning all maps first is practically a problem. Imagine all
> QGIS users waiting such long for every GRASS raster map...
> That's why we should find a better way.

What about the other option, i.e. just treating every map as if it
does have null cells?

> > Regarding the approach of adding a null flag to the cellhd file, I
> > suspect that it may be harder to get right than it might initially
> > appear. I.e. you could spend the next few years occasionally bumping
> > into cases where the flag ends up out of sync with reality.
> 
> Embedding the NULL values seems to be the best solution.

There may be a misunderstanding here. I didn't propose embedding the
nulls as a solution for this issue; I'm not sure how it would help. I
mentioned it as something which might happen for other reasons; in
which case, the proposed technique of testing for the presence of a
null file wouldn't work.

> Would it be complicated (in fact it might simplify the GRASS 
> raster lib)?

One unknown (for me) is whether XDR supports NaNs. The integer case
should be straightforward (AFAICT, we can use minus zero for null).
It would certainly simplify the library.

OTOH, there are a fair number of other things which could reasonably
be done to the raster library. IMHO, we should discuss all the issues
then re-write the raster I/O code from scratch.

Also, I do note that the files:

	src/libes/ogsf/Gs3.c
	src/raster/r.le/r.le.pixel/driver.c
	src/raster/r.le/r.le.pixel/cellclip.c

use G_get_null_value_row(). I have no idea why; application code
should probably just read the data and use G_is_null_value() etc.

-- 
Glynn Clements <glynn.clements at virgin.net>