[GRASS-dev] r.colors "-e" not going to full range

[Glynn Clements]

Hamish wrote:

> > > I just created a v.surf.rst elevation surface from some point data
> > > which has a few negative values in it. After creating the surface I
> > > set the colors with 'r.colors -e color=bcyr'. It looks really nice
> > > but there are white holes in the d.rast display.
> > > 
> > > r.info -r shows:
> > > min=-1.510814
> > > max=146.157700
> > > 
> > > The colr/ file doesn't go to one integer beyond the full range:
> > > ----------------------------------
> > > % -1 145
> > > nv:255
> > > *:255
> > > -1:0:0:255 0:0:94:255
> > > ...
> > > 144:255:1:0 145:255:1:0
> > > ----------------------------------
> > > 
> > > ??
> 
> Glynn:
> > I didn't realise that "r.colors -e" even worked on FP data;
> > "r.colors color=grey.eq" doesn't.
> > 
> > Ultimately, the code is dependent upon the categories returned by
> > G_next_cell_stat(). If it only returns up to category 145, then the
> > colour table will only extend to that value.
> 
> 
> > > min=-1.510814
> > > max=146.157700
> ..
> > > % -1 145
> 
> I could assume the -1.51 -> -1 problem was due to a cast to CELL when
> using a FP map with "*_cell_stat". The 146.15 vs 145 is a bit weirder,
> cast + missing the last iteration? No idea how to make it extend to be
> "% -2 147" for FP maps with G_next_cell_stat().

It's more complex than a simple cast. For computing statistics,
r.colors reads the map as CELL (as that's all that Cell_stats
supports). The map will be converted using G_quant_get_cell_value()
(lib/gis/quant.c).

> 
> > > I also noticed for another map 'r.colors -e' made a massive and slow
> > > colr/ file (1 for each cat) when the raster values were +-250000. That
> > > was a diff map centered on zero and I was using r.colors.stddev (wiki
> > > addons) to set the colors. While it had those huge outliers 95% of the
> > > points were +-5 (hence using the -e flag to get useful colors). It
> > > took impossibly long to render of course, so I looked in the colr/
> > > file and discovered it was huge.
> > > 
> > > Ok, maybe that's a feature of the -e method. But what I saw that was
> > > interesting was that between -265000 and -120 all the R:G:B values
> > > were identical, ie all those thousands and thousands of rules could
> > > have been condensed into a single "-265000:R:G:B -120:R:G:B". And
> > > that a little program to do that would probably be trivial.  I am a
> > > bit unsure if that condensing should happen as a addon script which
> > > reprocesses the file, when the colr/ file is created by 'r.colors
> > > -e', or when the color rules are read in upon d.rast. (or some
> > > combination of the above) It would seem like a nice enhancement to
> > > have that happen though.
> > > 
> > > I notice that even in my current -1.5 thru 146m map '-e' produces
> > > some redundant entries.
> > 
> > Can you test the attached patch?
> 
> for a random lidar test map, the colr/ file is 21349 lines long unpatched
> and 1497 lines long patched. Nice. There is still room for improvement
> though:
> 
> [color_xform.c unpatched]
> % -14900 1462400
> nv:255
> *:255
> -14900:0:0:255 -14100:0:0:255
> -14100:0:0:255 -14000:0:0:255
> -14000:0:0:255 -13800:0:0:255
> -13800:0:0:255 -13700:0:0:255
> -13700:0:0:255 -13600:0:0:255
> -13600:0:0:255 -13400:0:0:255
> -13400:0:0:255 -13300:0:0:255
> -13300:0:0:255 -13200:0:0:255
> -13200:0:0:255 -13100:0:0:255
> -13100:0:0:255 -13000:0:0:255
> -13000:0:0:255 -12900:0:0:255
> ...
> 
> [color_xform.c patched]
> % -14900 1462400
> nv:255
> *:25
> -14900:0:0:255 -10500:0:0:255
> -10500:0:0:255 -10400:0:1:255
> -10400:0:1:255 -9800:0:1:255
> -9800:0:1:255 -9750:0:2:255
> -9750:0:2:255 -9350:0:2:255
> -9350:0:2:255 -9300:0:3:255
> -9300:0:3:255 -9000:0:3:255
> -9000:0:3:255 -8950:0:4:255
> -8950:0:4:255 -8700:0:4:255
> -8700:0:4:255 -8650:0:5:255
> -8650:0:5:255 -8450:0:5:255
> -8450:0:5:255 -8400:0:6:255
> -8400:0:6:255 -8300:0:6:255
> -8300:0:6:255 -8200:0:7:255
> -8200:0:7:255 -8050:0:7:255
> -8050:0:7:255 -8000:0:8:255
> -8000:0:8:255 -7900:0:8:255
> -7900:0:8:255 -7850:0:9:255
> -7850:0:9:255 -7700:0:9:255
> ...
> 
> it seems that every second line is redundant.

In this case, yes, as only one component changes at each step, and
only changes by one. If the colour table has any discontinuities,
eliminating the flat sections would make a significant difference.

Ultimately, this should still be an improvement over the previous
output. For now, I'm more concerned with it being right than getting
further efficiency.

> aside: I hadn't noticed the *:color [undefined, does this correspond to
> "default" in G_parse_color_rule()?]

Yes.

> invert, and shift:n colr/ rules abilities before. invert is now
> covered by the r.colors -n flag using another method,

This is what r.colors -n uses. It calls G_invert_colors(), which
inverts the "invert" field in the Colors structure, which corresponds
to the "invert" keyword in the file.

> is 'shift' of any use?

I don't know. If it's present, all values are shifted by this amount
and (if necessary) wrapped into the range of the colour table before
they are looked up.

-- 
Glynn Clements <glynn at gclements.plus.com>