[GRASS-dev] Raster format and dual function module

[Glynn Clements]

Hamish wrote:

> > > > For the time being, it would be better to see if we can improve
> > > > the situation with some minor changes to a few key areas. It would
> > > > help if someone has the time to build GRASS with profiling support
> > > > and actually profile some common usage patterns.
> > > 
> > > You don't need to build with any special profiling support to try,
> > > (!)
> > > 
> > > Run from valgrind, save interim results to a file using the
> > > callgrind plugin, then visualize that file with KCachegrind.
> > > 
> > > see  http://kcachegrind.sourceforge.net/cgi-bin/show.cgi
> > 
> > In which case, it would be useful if someone could use that to analyse
> > some common usage cases.
> 
> could you write some precise/focused command line examples of "common"
> or likely intensive operations? (using spearfish)

No. I'm a programmer, not a geoscientist; that's why I'm asking
someone else to do it. Anything I came up with would be a contrived
example rather than "common usage". It would probably be more useful
if people could try it on "real" examples.

Having said that, the most interesting cases are ones where the
processing is relatively simple (so that the performance of libgis'
I/O routines is the dominant factor) and which tend to be slow (making
a 20-second operation take 10 seconds is more useful than making a
2-second operation take 1 second).

> > Cases which involve more than one process (e.g. d.*, or v.* when used
> > with a separate DBMS server) are harder to analyse.
> > 
> > If it turns out that the separate null file is a signficant
> > performance issue, we need to consider a migration plan for embedding
> > nulls (e.g. if 6.3 can write out rasters with embedded nulls, do we
> > need 6.2 to be able to read them?).
> 
> presumably FCELL and DCELL can use nan inline to identify NULL, CELL
> needs something else?

All of the data types have a defined null value which is stored in
internal buffers (by G_get_raster_row() etc). FWIW, the actual bit
patterns used internally are:

	CELL	0x80000000
	FCELL	0xFFFFFFFF
	DCELL	0xFFFFFFFFFFFFFFFF

[The FCELL/DCELL patterns are NaNs, but so is anything with an
all-ones exponent and a non-zero mantissa; G_is_[fd]_null_value checks
for the specific bit pattern. The CELL pattern is -2^31, i.e. the
value least likely to occur by accident.]

G_put_raster_row() checks each value with G_is_[cfd]_null_value(). 
For a null value, the value written out to the cell/fcell file is
zero, with a 1 written to the null file.

Conversely, G_get_raster_row() replaces the value read from the
cell/fcell file with the null value if the corresponding bit in the
null file is set.

There's no reason why the null value can't be written to or read from
the cell/fcell file. AFAIK, the current mechanism was used so that 5.x
maps would work with 4.x (which ignores the null file but treats zero
as null).

-- 
Glynn Clements <glynn at gclements.plus.com>