[pdal] Quit copying yourself

[Howard Butler]

All,

Andrew Bell has been revisiting some of the architectural choices of PDAL after recently implementing a reasonably complex driver. This email is a head's up about some planned changes we've discussed that are coming to PDAL. We hope they will make things more simple, allow us to chuck a lot of things that might be beside the point, and make it easier for someone to learn a fewer bits and pieces to be able to make progress developing software with the library. 

The way we thought we'd use PDAL when Michael Gerlek started laying things out, and how it's actually used today are a little bit different. One area where this incongruity has become apparent is the limitations of using PointBuffer as the substrate for moving data back and forth throughout PDAL. The PointBuffer as it is currently implemented assumes a data storage arrangement in addition to an access pattern, and this causes Stages to have to move/copy data around to satisfy the requirements imposed by the PointBuffer when a Stage looking to rearrange data. 

To alleviate this pressure, we going to hide the data layout and storage of the PointBuffer behind a PointContext. Raw byte mucking with the PointBuffer is going to be discouraged or possibly not allowed anymore. In exchange for this limitation, Stage implementors will be able to interact with a single PointContext that hides the data storage and allows you to make PointBuffers that are simply views. No more making little PointBuffers, copying them onto bigger PointBuffers, or using hard-to-use and not-implemented-everywhere randomIterator to skip/hop around in the Stage. A filter like filters.chipper or filters.tiler will be able to simply chop up the data into a bunch of smaller PointBuffers, organizing the data however they want, with each simply being a view into the single PointContext.

If you have your own code that does a lot of composition of PDAL major objects, we are interested in hearing about it. Note that the pipeline XML stuff shouldn't have to change at all.

Howard