<div dir="ltr">Perhaps it would help if you posted some code.<div><br></div><div>I think I understand what you're generally trying to achieve.</div><div><br></div><div>The chipper filter can't use streaming mode because it requires sorting of the points in order to determine cells. The splitter filter is a more traditional tiling filter, and it could, theoretically, use streaming mode (since points are binned by nothing than their location). The issue is that there is no "routing" in the pipeline when in streaming mode. There is no way to say "send this point on to stage X". Or any generic way to have a downstream stage X realize that the point needs to be processed in such and such a way. Perhaps we could think about doing something like this, but it's not generally doable right now.</div><div><br></div><div>I'm not sure what your splitter is doing, but if it were to tag the points with some sort of cell ID, you could make a single writer stage accepts points from your splitter that essentially multiplexes the points based on that cell ID. Or you could have your splitter simply do the writing based on the cell for each point.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 18, 2017 at 10:40 AM, Jason Beverage <span dir="ltr"><<a href="mailto:jasonbeverage@gmail.com" target="_blank">jasonbeverage@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey Andrew!<div><br></div><div>Thanks for getting back to me. I think part of the issue is that I don't quite understand how to convert my current cursor based LASTools code to a streaming Stage based system.</div><div><br></div><div>I've attached an image of what I'm trying to do.</div><div><br></div><div>Basically I want to have a reader that can stream a bunch of points in, then a splitter stage that determines which of 9 potential writers to add the point to based on the location of the point and some other criteria, but I want to do it all without storing all of the points in memory.</div><div><br></div><div>The existing ChipperFilter is similar to what I want to do, but it doesn't appear to support streaming. It looks like it does the partitioning in memory and then returns a PointViewSet with all of the outputs.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Jason</div></font></span><div><div class="h5"><div><br></div><div><br></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jul 18, 2017 at 10:18 AM Andrew Bell <<a href="mailto:andrew.bell.ia@gmail.com" target="_blank">andrew.bell.ia@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">On Mon, Jul 17, 2017 at 10:29 PM, Jason Beverage <span dir="ltr"><<a href="mailto:jasonbeverage@gmail.com" target="_blank">jasonbeverage@gmail.com</a>></span> wrote:</div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi all,<div><br></div><div>I'm new to PDAL and I'm currently porting an existing point cloud tookit we have at Pelican Mapping to use PDAL instead of LASTools and I had a couple of questions.</div><div><br></div><div>Our toolkit is similar to Entwine in that it takes a large number of input points from LAS/LAZ files and chops them up into tiled LAZ files with the goal of displaying them in OpenSceneGraph and osgEarth based applications.</div><div><br></div><div>Since we're dealing with a huge number of points, I'm making heavy use of streaming so I can avoid keeping the entire dataset in memory.</div><div><br></div><div>For reading, I found the StreamCallbackFilter so I can intercept the points coming from a stage and process them one by one as I was doing before, so I think that will work for my case.</div><div><br></div><div>For writing however, I'm running into a bit of snag probably due to my lack of understanding of the streaming API.</div><div><br></div><div>We basically have an algorithm that does something like this:</div><div><br></div><div>for each point in all the input points:<br></div><div> // Figure out which output file to write the point to based on it's location.</div><div> Writer* writer = getWriter(point);</div><div> writer->writePoint(writer);</div><div><br></div><div>So we create a bunch of output writers (using LASTools) based on an octree split, and for each point we determine which writer to write to and write the incoming point to the appropriate output file.</div><div><br></div><div>Is there a simple way to write points interactively to a Writer? It seems like if the write functions of Writer were public that it would be fairly easy to implement something like that.</div></div></blockquote><div><br></div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>I'm not sure I understand exactly. You can create your own stage that implements processOne() and do anything you like in it. There is nothing particularly special about a writer. It's just a stage. There's no reason you can't chain a bunch of StreamCallbackFilters together if that's helpful -- the first one could do whatever you're doing now, the second one could do the writing you want to do. Or they could be combined.</div><div><br></div><div>Is part of what you are wanting is to use PDAL's LAS writer instead of LAStools?</div><div><br></div></div></div></div><div dir="ltr"><div class="gmail_extra">-- <br><div class="m_2358538958081221983m_1821188321884791268m_-7008113717705040360gmail_signature" data-smartmail="gmail_signature">Andrew Bell<br><a href="mailto:andrew.bell.ia@gmail.com" target="_blank">andrew.bell.ia@gmail.com</a></div>
</div></div></blockquote></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Andrew Bell<br><a href="mailto:andrew.bell.ia@gmail.com" target="_blank">andrew.bell.ia@gmail.com</a></div>
</div>