[GRASS-dev] Using layers and categories for something else than ID and class

[Vaclav Petras]

[Was: v.in.lidar: store return or class as category]

Changing the subject as this is more general than just v.in.lidar.


On Wed, Sep 30, 2015 at 4:20 AM, Moritz Lennert <
mlennert at club.worldonline.be> wrote:

> On 29/09/15 19:49, Vaclav Petras wrote:
>
>>
>> On Tue, Sep 29, 2015 at 9:52 AM, Moritz Lennert
>> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
>> wrote:
>>
>>     On 29/09/15 15:26, Newcomb, Doug wrote:
>>
>>         As an example, the new North Carolina data set ( 2014 +) , the
>> LiDAR
>>         classes above 12 are being used for roads, (13) and bridges (14) .
>>
>>
>>     Maybe if we go for 30 or 31 we have less chance to step on other
>>     classifications' toes. But the risk remains, so maybe the idea of
>>     regrouping 0 and 1 as 1 is less error prone...
>>
>>
>> 0+1 > 1 sounds like a good idea and reasonable default.
>>
>> I will actually this again with color.
>> It seems reasonable to store the
>> color for vector points (if there is a lot of them) as categories.
>>
>
> Is writing a color table (à la v.colors) so inefficient that you want to
> put the colors in categories.
>

I did no benchmark, but I suppose that setting for case of lidar data with
colors (lots of point and each point has its own RGB) text color table
won't cut it unless we employ some reduction as in case of r.composite [1].

I have the feeling we are starting to abuse the concept of categories, here,


Maybe, but maybe not. I don't think that we've defined how layers and
categories should be used. Using categories in layer 1 as user-visible
feature IDs and categories in layer 2 as class(ification) is an obvious use
case, but I don't think we should limit ourselves to this one. After
realizing what information I can potential store as layers and categories,
I think that layers and categories are heavily underused. The system has a
lot of potential and there is just few cases when it is used fully (e.g.,
with temporal framework one can store different positions of a point in one
vector map but mark the individual time steps using layers).

Another point of view is that categories used as IDs are useless for lidar
(since you can't work with the attribute table anyway) and there might not
be enough of them (hitting limit of 32bit signed integer). So, the solution
is to not use categories at all (and loose all info about the point except
for xyz and possible processing), or to use categories for something else.
Return number and class are low hanging fruit here. RGB requires more
support but it seems feasible to me.

just because attribute management is slow. If that is the case, maybe we
> should rather look at improving performance of that ?


That's right it is slow. For the lidar case prohibitively slow. I don't see
an easy way how it could be faster. Different attribute storage backends
such as PostgreSQL may work better but still you have two separated
storages for coordinates and attributes which means two searches (that's a
naive view but I think it works). Then perhaps alternative drivers are the
way. PostGIS might be but perhaps a new vector driver such as PDAL might be
more appropriate (and is advantageous also for other reasons). However, all
this requires significant development effort (new driver, optimization) or
increases demand for user infrastructure (running a db server).

But
>> of course the range is 0-255, we can surely move it to 1-256
>>
>
> Why would you want to ?


If we want to store RGB as category and we don't want to to use category 0,
we need to store 0 (e.g., white, 0:0:0) somehow. Adding 1 to all color
values solves the issue as long as we subtract them when reading them for
the actual rendering.

but it
>> little bit inconsistent with everything else in world regarding the RGB
>> color handling.
>>
>
> RGB is 0-255 for each of the three colors, non ?


Yes. 256 numbers for each color in three layers. So you would specify (e.g.
in d.vect):

...red_layer=1 green_layer=2 blue_layer=3...

Other option is to combine RGB to one integer (using bit shifts), so it
would be stored as one category, so

...rgb_layer=1...

The need to shift from category 0 is in both cases. So we would always
subtract or add one before/after writing the category. Advanced users may
appreciate a flag to store the GRASS-invalid but common/expected range of
values.

[1] https://grass.osgeo.org/grass71/manuals/r.composite.html


>
>
>
> Moritz
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20150930/d971db09/attachment-0001.html>