[GRASS-dev] [grass-code P][372] scritps for converting raster maps into GRASS 7 format, and back

[Soeren Gebbert]

Hi,

Am Mittwoch, 18. April 2007 17:59 schrieb Glynn Clements:
> Paul Kelly wrote:
> > Although I still don't think I like the idea of changing the internal
> > raster format as I expect it will break lots of 3rd-party software that
> > reads GRASS raster files directly, and could make GRASS look bad for
> > changing things.
>
> FWIW, I don't feel that the future of GRASS should be constrained by
> whether or not third parties are willing to maintain compatibility.
>
> > E.g. thinking of the detailed discussion of the raster
> > format a year or two ago for JavaGrass which reads it directly I think.
> > Also even GDAL must be passed the path to the cellhd directory to read a
> > GRASS raster I think? Also might it not be worth taking the opportunity
> > to modernise the raster format further than just re-arranging the files,
> > and perhaps providing some kind of LGPL library for developers of
> > 3rd-party software to use to read GRASS rasters (actually that already
> > exists I think - isn't it what GDAL uses?
> > http://freegis.org/cgi-bin/viewcvs.cgi/libgrass/ but don't think its used
> > much.)
>
> Actually, I've been thinking about hiving off the base raster format
> to a separate library, along with generalising the low-level I/O to
> allow for an "r.external" command, i.e. allow GRASS raster I/O to
> directly read external files through GDAL.
>
> The higher level stuff (rescaling, format conversion, reclassing, MASK
> etc) would remain in GRASS.

That is an interesting approach. But we already have a very sophisticated  
library in grass, which can be used for raster data management. 
Im talking about the g3d library. IMHO this lib can be extended to support 
raster maps.

The g3d lib supports state of the art data access. 
I recently readed a terralib document 
about handling raster data and noticed that the g3d library features are quite
progressive.

It supports:
* variable tile sizes
* direct uncached tile access
* cached random value access
** while random acces, the tiles which are accessed last are kept in the 
   cache memory  
** support of different data type sizes
* the directory structure is of type g3d/map_name/files

We should consider to extent this lib for internal usage. There is no problem
to add an abstraction layer to support external raster map implementations,
like the callback design in the display driver code. 

Some kind of OO design would be great:

example:

typedef struct G2d_data {

  Data_source *map; /*g3d or what ever*/
  Metadata stuff .... ;

  /*value access member functions*/
  CELL (*get_c_value)(struct G2d_data*, int, int);  
  FCELL (*get_f_value)(struct G2d_data*, int, int); 
  ...
  void (*put_c_value)(struct G2d_data*, int, int, CELL);
  void (*put_f_value)(struct G2d_data*, int, int, FCELL);
 ...

  /*tile access meber functions*/
  CELL *(*get_c_row)(struct G2d_data*, int, int);
  CELL *(*get_c_tile)(struct G2d_data*, int, int);
  ...
  int (*put_tile)(struct G2d_data *, int, int);  
  ...
} G2d_data;

The callback member functions are static functions like this:

G_put_c_value(G2d_data* map, int col, int row, CELL value);
and so on ...
And set while opening a raster data source.

A simple data acces should look like this:

{
G2d_data *data_source;
CELL value;

data_source = G_open_raster_data_source(name, type, ...);

value = data_source->get_c_value(data_source, 3, 4);
data_source->put_c_value(data_source, 3, 4, value);
}

I will upadte the gpde library with this kind of member function data access.
I hope we consider to extent the g3d lib and additionally add new metadata 
handling and temporal data management.

Best regards
Soeren

>
> The main issue is that some modules perform random access (i.e. don't
> read the rows in top-to-bottom order), and not all raster formats can
> support this efficiently (uncompressed formats such as BMP or raw-PPM
> can, but anything which uses compression needs an index).
>
> A related problem is that modules don't have to explicitly request
> non-linear access at open time, so there's no simple way to identify
> which modules would be affected.