[GRASS5] Raster files suggestion: new directory layout

[Glynn Clements]

Markus Neteler wrote:

> another proposal: Let's simplify the raster file directory layout
> and have all files related to a raster map staying in a single
> directory (such as the 5.7 vector files). We already have discussed
> it some time ago.

Agreed. But I would suggest doing much more than simply moving the
files around. Off the top of my head, reasonable (IMHO) possibilities
include:

- Write raw C int/float/double rather than using XDR (FP) and
  home-grown (integer) encodings.

[Makes portability slightly harder; we would need to be able to handle
both big- and little-endian files, and possibly provide IEEE FP
encoding/decoding for architectures which don't use IEEE natively. But
it might provide a big performance boost if the typical-case I/O path
is to simply call read() or write(); or even just mmap().]

- Use zlib for compressing both integer and FP maps.

- Support 1/2/4 bits-per-cell formats.

- Embed nulls rather than using a separate null file.

- Support fixed-point format; i.e. the raw data is integers, but the
  header contains FP slope and offset values with which the data is
  transformed, so it appears as FP to the application. This would
  effectively provide 1/2/4/8/16 bits-per-cell FP maps.

- Support "rescale" maps; a cross between fixed-point maps and reclass
  maps. Like a reclass map but the transformation consists of
  slope and offset coefficients rather than a list of discrete
  categories. Unlike reclass maps, this would work for FP maps (and
  integer maps with many categories).

- Incorporate rowio and/or segment functionality directly. Possibly
  generalise map format to rectangular tiles (a traditional row-by-row
  map would just have tiles which are one row high and as wide as the
  map).

[Ideally, I would like the ability to write GET(map_fd, row, col) and
PUT(map_fd, row, col, value), and have it work efficiently, and not
have to bother with {get,put}_row() operations at all.]

- Incorporate g3d functionality directly (a 2D map would just be one
  "slice" deep).

> Advantages:
> - clean and understandable file structure
> - cleanup of raster API which is hard to understand at time

I don't think that the existing API is particularly difficult
(although there is a lot of redundancy). The *implementation*, OTOH,
can be extremely hard to understand.

> When to implement it:
> - I suggest to do it asap in GRASS 5.7. While at migrating
>   their vector databases the users can also migrate the raster
>   databases (which is simply a 'mv' of the files).

I suggest spending some time thinking about it first. The raster I/O
code constitutes the foundation for half of GRASS (vectors being the
other half). It would help to get it right at the outset.

-- 
Glynn Clements <glynn.clements at virgin.net>