[GRASS-dev] G_put_raster_row and OpenMP

[Glynn Clements]

Yann wrote:

> I am trying to include OpenMP coding into raster processing so that each row 
> can be processed by a different process.
> 
> The problem faced is that while row number 1 and 2 are still processing, row 
> number 3 may already have an ouput for wirting in the output file. Therefore 
> we would need to specify the row number to write the ouput row to.
> 
> Right now,
> 
> int G_put_raster_row (int fd, const void * buf, RASTER_MAP_TYPE data_type)
> 
> does not include row number specification. So I tried to find other functions 
> in put_row.c:
> 
> int G_put_map_row_random (int fd, const CELL * buf, int row, int col, int n)
> 
> This function is giving control to the data (row,col) location. However it 
> writes only CELL type of data, which is limiting a lot what is possible to do 
> with it in remote sensing processing.
> 
> Ideally, what we would be looking for is a G_put_raster_random_row() function, 
> giving only the row input location would be fine actually, since distributed 
> row processing is faster than pixel-wise.
> 
> I dont know what are the plans or interests to produce raster code that goes 
> this way, but this is something simple I believe would help OpenMP coding.
> 
> If people dont want to do it, I could try doing it.
> 
> If I missed a major concept, please do not hesitate to correct me :-)

I don't know if it's "major", but one thing to bear in mind is that
the map must have been opened with G_open_cell_new_random() in order
to be able to use that function.

This has several consequences: the map is stored uncompressed, it
cannot contain nulls, and it cannot be floating-point (essentially,
when FP and null values were added in 5.x, the random-access case
wasn't upgraded).

Neither of these factors would be particularly hard to change.

The raster file contains a table of row pointers, so the rows don't
actually have to be stored in sequence in the file, even if they're
compressed. They could be written out in an arbitrary order (although
not concurrently).

The null bitmap isn't compressed, so it would be possible to write out
rows in an arbitrary order, or even concurrently. The existing code
can't handle this because it caches a block of consecutive rows and
writes them all out at once. Changing this would introduce
inefficiency, as the null file is opened and closed for each write (to
avoid doubling the number of open file descriptors).

-- 
Glynn Clements <glynn at gclements.plus.com>