[pdal] Array Dimensions

[Howard Butler]

On Jul 1, 2013, at 9:54 AM, Chris Foster <chris.foster at roames.com.au> wrote:

> On 1 July 2013 23:34, Howard Butler <hobu.inc at gmail.com> wrote:
>> 
>> Were you planing to use PDAL for something other than data translation?
> 
> I have two uses in mind.  One is to extract chunks of data from a custom point
> database which we have at work.  We already have a filtering system but pdal
> is much slicker, particularly the pipeline serialization side of things, which
> is a real pleasure to behold :)  

I'm glad you've found it useful. It is a shameless rip-off of GDAL's VRTs (in spirit).

> Ultimately I've been writing sophisticated
> filter chains where some stages require building spatial datastructures from
> the points.  The filters in my current code aren't composable however, and you
> can't add custom dimensions to communicate between stages.  I'm hoping that
> using the pdal API will solve those problems.  

> It seems to me that requiring
> spatial context in a filter goes a bit beyond what pdal was designed for, but
> given you do everything in chunks I'm hoping it's possible without too much
> effort.

I've been thinking about this one too. PointBuffer does carry its "bounds" around, which I've used 
for some communication (mostly going in/out of database backends), but we haven't hinted the entire pipeline 
with spatial awareness.

Additionally, there's a FLANN-based filter available, and that's not plugged into everything like it should.
I don't know that I want to make FLANN a hard dependency, but I've found nanoflann, which is an 
all-in-one-header port of BSD-licensed FLANN, so that could be a possibility now. I'd be interested 
in any ideas you have to make this aspect better.


> The other use is as a reader for various point cloud formats (primarily .las
> and .laz) for my point cloud viewer displaz (https://github.com/c42f/displaz).
> I integrated initial (very basic) support for pdal on the weekend.  Displaz is
> meant to be a swiss army knife for geospatial point cloud visualization, so it
> needs the ability to read and display arbitrary point dimensions, and good
> metadata support.  Currently I have a quick and dirty PointArray to hold the
> data in memory, but I'd like to replace this with pdal's PointBuffer if
> possible.  I really want flexible visualization of arbitrary point dimensions
> whose names and types are not known at compile time.

Sounds great.

> Right.  Question: what's the distinction between UnsignedByte and
> UnsignedInteger?  It's confusing to me to have both of these.

*interpretation* is the keyword here, not merely type. Do I interpret this dimension as byte or integer? The idea was that I might care about the difference. UnsignedByte isn't used much though, and it could be removed without consequence if it is clear this just makes things confusing.

> Arrayness seems orthogonal from the element type to me.  In the past I've
> flagged arrays simply by having element count > 1.

The getInterpretation is to be able to discriminate how we should treat a dimension.

again, interpretation, not type. So for boost::array<float, 9>, we'd have

getInterpretation() == dimension::Float
getElementCount() == 9
getByteSize() == 4*getElementCount()

> 
>> * pdal::Dimension::getByteSize() needs to be made dynamic when the dimension type is array
> 
> I assume this doesn't come with a performance hit?  I'm just wondering if
> there's code somewhere which habitually checks the byte size in a tight loop,
> expecting it to be a simple memory access.

ah, good point. It is not beneath me to simply cache the value once it has been calculated for a dimension instance, however.

> Another more wacky thought which crosses my mind is that it would be helpful
> to have aliases (eg, "Red", "Blue" and "Green" dimensions alias fields of the
> "Color" dimension which is a three element array per point).  This would be
> rather handy for me since I actually don't care to read the color components
> separately, but it might be too hard to get right.

/me strokes his chin and wonders how messy dimension aliasing could get.

I'm not against it, mind you, but I'm just a bit worried that the dimension/schema stuff is a bit messy as it is.  
Pull up a chair and let me tell you about dimension inheritance when you have the time... :)