[GRASS5] the floating raster file format

[Markus Neteler]

On Wed, Apr 14, 2004 at 07:17:22AM +0100, Glynn Clements wrote:
> 
> Andrea Antonello wrote:
> 
> > after a good bottle of Montpulciano D'abruzzo and seven hours (9pm Saturday to 
> > 4am Sunday) in the pre-eastern night, with the help of my good friend "el 
> > Pitero", the amazing Khexedit, the Grass sources and the r.compress command, 
> > we finally have the binary raster format of floating point maps in Grass. 
> > I'ld like to share that, since apart of the bottle of good wine it was a 
> > heavy experience and none should repeat it:). I hope somebody can find 
> > usefull what follows and give me hints or make me notice possible errors.
> > 
> > Alright, let's get started:
> > My explanation example is a 3x3 floating point file. 
> > At the begin of the file we have 4 bytes, These are the same for compressed, 
> > uncompressed, integer or floating, so I left them aside for now. After these 
> > 4 bytes, other three pairs of 4 bytes follow, which contain the address of 
> > each row of data in the file (this also tells us, that there is a limit in 
> > the dimension of raster files, even if huge). After these bytes the rows of 
> > data start. Each row can be simply passed through a zlib decompressor. After 
> > that action we have the row of data in uncompressed format. The only thing 
> > left to do is to get out of that stream the right dataformat. That's all, 
> > rather simple if you know how to do it :)
> > 
> > With the help of a small example:
> > 
> > The file is the following 3x3 floating point matrix:
> > 10.000 20.000 30.000
> > 20.000 40.000 50.000
> > 30.000 50.000 60.000
> > 
> > Here is the binary and decimal version:
> > THE HEADER PART:
> > binary:    00000100 00000000 00000000 00000000 
> > decimal: 4              0                0               0                
> > 
> > 00010001 00000000 00000000 00000000 
> > 17             0               0               0        
> > 
> > 00100100 00000000 00000000 00000000 
> > 36             0               0               0        
> > 
> > 00110111 00000000 00000000 00000000 
> > 55            0               0               0        
> > 
> > 01001010 00110001 
> > 74            49
> > 
> > COMMENTS:
> > 4 could be the number of bytes used to define the address of each row, but I 
> > would not bet about that.
> > 17 is the adress in bytes after which the first row of data starts,
> > 36 is the same for the second row and 55 for the third. 74 is the end of file.
> > I noticed the last byte only now, I will try to understand it later.
> > 
> > After that, the rows of data come and since we know the offset between the 
> > rows, we know exactly how many bytes to send through the zlib decompressor.
> > 
> > 
> > Any comment or hint is very appreciated.
> 
> The above is incorrect.
> 
> The header is a single byte, equal to sizeof(long) (typically 4 on a
> 32-bit platform, 8 on a 64-bit platform). Then, NROWS+1 offsets are
> written as longs (i.e. 4 or 8 bytes, depending upon platform) in
> big-endian (Motorola) byte order.
> 
> Thus, your example is actually interpreted as:
> 
> 	4		sizeof(long)
> 	0 0 0 17	offset of row 0
> 	0 0 0 36	offset of row 1
> 	0 0 0 55	offset of row 2
> 	0 0 0 74	offset of end of data
> 
> See G__write_row_ptrs() in src/libes/gis/format.c for the code which
> writes this data. However, note that the row offsets are initially
> zero; they get overwritten later (if you're writing compressed data,
> you don't know how much space it will require until you've compressed
> it).
> 
> As for the format of the actual row data, see put_fp_data() in
> src/libes/gis/put_row.c and RFC 1014 (the XDR specification).

I have taken the liberty to write these
comments into
 src/libes/gis/format.c

Please update there if needed.

Note: The XDR RFC 1014 is found at:
 http://www.faqs.org/rfcs/rfc1014.html

Markus