[GRASS-dev] Raster format and dual function module

[Hamish]

> > > > > For the time being, it would be better to see if we can
> > > > > improve the situation with some minor changes to a few key
> > > > > areas. It would help if someone has the time to build GRASS
> > > > > with profiling support and actually profile some common usage
> > > > > patterns.
> > > > 
> > > > You don't need to build with any special profiling support to
> > > > try, (!)
> > > > 
> > > > Run from valgrind, save interim results to a file using the
> > > > callgrind plugin, then visualize that file with KCachegrind.
> > > > 
> > > > see  http://kcachegrind.sourceforge.net/cgi-bin/show.cgi
> > > 
> > > In which case, it would be useful if someone could use that to
> > > analyse some common usage cases.
> > 
> > could you write some precise/focused command line examples of
> > "common" or likely intensive operations? (using spearfish)
> 
> No. I'm a programmer, not a geoscientist; that's why I'm asking
> someone else to do it. Anything I came up with would be a contrived
> example rather than "common usage". It would probably be more useful
> if people could try it on "real" examples.
> 
> Having said that, the most interesting cases are ones where the
> processing is relatively simple (so that the performance of libgis'
> I/O routines is the dominant factor) and which tend to be slow (making
> a 20-second operation take 10 seconds is more useful than making a
> 2-second operation take 1 second).

so a good candidate for a high libgis IO module + slow task might be,

(spearfish60)
g.region elevation.10m
r.patch out=merge in=roads,railroads,streams,tractids,transport.misc,rstrct.areas,trn.sites,landuse,fields,erode.index

or the r.mapcalc equivalent of the same task?

As "common" tasks are very different for everyone, it is hard to define.


> > > Cases which involve more than one process (e.g. d.*, or v.* when
> > > used with a separate DBMS server) are harder to analyse.
> > > 
> > > If it turns out that the separate null file is a signficant
> > > performance issue, we need to consider a migration plan for
> > > embedding nulls (e.g. if 6.3 can write out rasters with embedded
> > > nulls, do we need 6.2 to be able to read them?).
> > 
> > presumably FCELL and DCELL can use nan inline to identify NULL, CELL
> > needs something else?
> 
> All of the data types have a defined null value which is stored in
> internal buffers (by G_get_raster_row() etc). FWIW, the actual bit
> patterns used internally are:
> 
> 	CELL	0x80000000
> 	FCELL	0xFFFFFFFF
> 	DCELL	0xFFFFFFFFFFFFFFFF
> 
> [The FCELL/DCELL patterns are NaNs, but so is anything with an
> all-ones exponent and a non-zero mantissa; G_is_[fd]_null_value checks
> for the specific bit pattern. The CELL pattern is -2^31, i.e. the
> value least likely to occur by accident.]
> 
> G_put_raster_row() checks each value with G_is_[cfd]_null_value(). 
> For a null value, the value written out to the cell/fcell file is
> zero, with a 1 written to the null file.
> 
> Conversely, G_get_raster_row() replaces the value read from the
> cell/fcell file with the null value if the corresponding bit in the
> null file is set.
> 
> There's no reason why the null value can't be written to or read from
> the cell/fcell file. AFAIK, the current mechanism was used so that 5.x
> maps would work with 4.x (which ignores the null file but treats zero
> as null).

This part of the next raster format seems clear then.


Hamish