[pdal] Reading and Writing outside of a stage

[Jason Beverage]

Hey Andrew!

Thanks for getting back to me.  I think part of the issue is that I don't
quite understand how to convert my current cursor based LASTools code to a
streaming Stage based system.

I've attached an image of what I'm trying to do.

Basically I want to have a reader that can stream a bunch of points in,
then a splitter stage that determines which of 9 potential writers to add
the point to based on the location of the point and some other criteria,
but I want to do it all without storing all of the points in memory.

The existing ChipperFilter is similar to what I want to do, but it doesn't
appear to support streaming.  It looks like it does the partitioning in
memory and then returns a PointViewSet with all of the outputs.

Jason



On Tue, Jul 18, 2017 at 10:18 AM Andrew Bell <andrew.bell.ia at gmail.com>
wrote:

> On Mon, Jul 17, 2017 at 10:29 PM, Jason Beverage <jasonbeverage at gmail.com>
> wrote:
>
>> Hi all,
>>
>> I'm new to PDAL and I'm currently porting an existing point cloud tookit
>> we have at Pelican Mapping to use PDAL instead of LASTools and I had a
>> couple of questions.
>>
>> Our toolkit is similar to Entwine in that it takes a large number of
>> input points from LAS/LAZ files and chops them up into tiled LAZ files with
>> the goal of displaying them in OpenSceneGraph and osgEarth based
>> applications.
>>
>> Since we're dealing with a huge number of points, I'm making heavy use of
>> streaming so I can avoid keeping the entire dataset in memory.
>>
>> For reading, I found the StreamCallbackFilter so I can intercept the
>> points coming from a stage and process them one by one as I was doing
>> before, so I think that will work for my case.
>>
>> For writing however, I'm running into a bit of snag probably due to my
>> lack of understanding of the streaming API.
>>
>> We basically have an algorithm that does something like this:
>>
>> for each point in all the input points:
>>     // Figure out which output file to write the point to based on it's
>> location.
>>     Writer* writer = getWriter(point);
>>     writer->writePoint(writer);
>>
>> So we create a bunch of output writers (using LASTools) based on an
>> octree split, and for each point we determine which writer to write to and
>> write the incoming point to the appropriate output file.
>>
>> Is there a simple way to write points interactively to a Writer?  It
>> seems like if the write functions of Writer were public that it would be
>> fairly easy to implement something like that.
>>
>
> I'm not sure I understand exactly.  You can create your own stage that
> implements processOne() and do anything you like in it.  There is nothing
> particularly special about a writer.  It's just a stage.  There's no reason
> you can't chain a bunch of StreamCallbackFilters together if that's helpful
> -- the first one could do whatever you're doing now, the second one could
> do the writing you want to do.  Or they could be combined.
>
> Is part of what you are wanting is to use PDAL's LAS writer instead of
> LAStools?
>
> --
> Andrew Bell
> andrew.bell.ia at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20170718/8422bb09/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: splitter.jpg
Type: image/jpeg
Size: 40626 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20170718/8422bb09/attachment-0001.jpg>