[GRASS-dev] Raster format and dual function module

[Joel Pitt]

Hi all,

I've been thinking abit about GRASS and the two things which I'd like
to see change. However I *don't* think that they these things should
be changed just for me and recognise the need to remain compatible
with existing modules etc.

1) Raster file format - Unless I'm mistaken, the raster is a flat
compressed file, with NULL values stored uncompressed in a seperate
bitmask. This means the NULL mask is quite large and also means the
raster is split across files. Obviously this makes it fastish, but
large and sparse maps are unnecessarily huge.

I've also been frustrated that one has to read an entire set of rows
to find out whether there are any cells Non-null. This makes modelling
processes slower then they need to be. I think that a quadtree format
would solve this (lower resolution versions of an region overlayed on
one another e.g. 128x128 64x64 32x32) by allowing you to descend only
those tree branches that have raster cells present. Quadtrees would
also speed up the process of displaying large raster maps on limited
resolution monitors.

Quadtrees based rasters with integrated null bitmasks could be easily
accessed through the normal raster function calls ensuring existing
modules are interoperable. Only new modules wanting to use quadtree
functions (such as checking existing of values at positions higher up
the tree) woudl have to check the version of the GRASS raster.

2) Dual function modules - there is alot of talk about how we want to
move forward with the GUI but still maintain the seperate programs for
GRASS commands. I suggest having a compilation system that will create
both standalone executables and integrated libraries. A GUI could then
check whether a library version of GRASS module exists and load that,
or otherwise use the equivalent executable.

This is due to another frustration I've had with speed of execution.
While running long chains of commands in simulations I can't help
realising that every command has to reload a map from disk and then
write it back. As we all know, disk access is REALLY slow in
comparison to memory etc. so if GRASS modules were compiled as
libraries the last N (N being configurable) number of loaded maps
could remain in memory for quick processing and display.

Let me know what people think. Obviously I don't expect other people
to go ahead with this just because I would like to see it, but if
people see some value in these approaches I could attempt to map out a
course and contribute some time to it.

Cheers,
Joel

-- 
"Wish not to seem, but to be, the best."
                -- Aeschylus