<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
Andrew,</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
Appreciate the explanation. </div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: EmojiFont,Calibri,Helvetica,sans-serif,'Apple Color Emoji','Segoe UI Emoji', NotoColorEmoji,'Segoe UI Symbol','Android Emoji',EmojiSymbols; font-size: 12pt;">
It would be useful to understand why this issue occurred. Does appear to be reproducible
<span>😊</span></div>
<div style="color: rgb(0, 0, 0); font-family: EmojiFont,Calibri,Helvetica,sans-serif,'Apple Color Emoji','Segoe UI Emoji', NotoColorEmoji,'Segoe UI Symbol','Android Emoji',EmojiSymbols; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: EmojiFont,Calibri,Helvetica,sans-serif,'Apple Color Emoji','Segoe UI Emoji', NotoColorEmoji,'Segoe UI Symbol','Android Emoji',EmojiSymbols; font-size: 12pt;">
Regards, Lars</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
</div>
<hr tabindex="-1" style="width: 98%; display: inline-block;">
<div id="divRplyFwdMsg" dir="ltr"><font color="#000000" face="Calibri, sans-serif" style="font-size: 11pt;"><b>Fra:</b> Andrew Bell <andrew.bell.ia@gmail.com><br>
<b>Sendt:</b> onsdag 12. september 2018 15.41<br>
<b>Til:</b> Lars<br>
<b>Kopi:</b> pdal@lists.osgeo.org<br>
<b>Emne:</b> Re: [pdal] Python and chipper filter</font>
<div> </div>
</div>
<meta content="text/html; charset=utf-8">
<div>
<div dir="ltr">The chipper filter creates sets of points for each tile. These sets (called point views) pass through the pipeline individually and are processed separately by each downstream filter. In your case, the chipper should create two or three point
views (since the count limit on the points is 10,000 and the capacity limit on the chipper is 5000 points). This essentially means that the python filter gets invoked two (or three) times. When you reverse the order of the python filter and the chipper,
the python filter is only invoked once, since at that point in the processing, only one point view exists.
<div><br>
</div>
<div>All that said, the differences you're seeing seem extreme and I think it would take more investigation to figure out the root cause of the timing differences.</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr">On Mon, Sep 10, 2018 at 3:47 AM Lars <<a href="mailto:laasunde@hotmail.com">laasunde@hotmail.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;">
<div dir="ltr">
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
Hello,</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
We have created a pipeline with a python script but are seeing some unexpected behaviour. The python script currently does nothing to exclude the script as culprit.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
Running the script with 10k count and pipeline is completed in less than a seconds. Increasing to 50k count and it takes to 2 seconds but increasing to 80k count suddenly it takes 150 seconds!!! Attempting to use 90k count and after 25 minutes it is still not
finished (pdal is constantly using 12,5% --> one core).</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
If we changed the order of the filters (python first and then chipper), then 10k count took less then a second and 100k takes 4 seconds.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
If we remove the python filter then 10k count took less then a second and 100k takes 4 seconds.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
This appears to be related to the python filter.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
Why is the the attached pipeline so slow? Is this a software issue or poorly designed pipeline?</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
Using PDAL 1.7.2 on Windows 10.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-serif; font-size: 12pt;">
kind regards, Lars<br>
</div>
</div>
_______________________________________________<br>
pdal mailing list<br>
<a href="mailto:pdal@lists.osgeo.org" target="_blank">pdal@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/pdal" target="_blank" rel="noreferrer">https://lists.osgeo.org/mailman/listinfo/pdal</a></blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="x_gmail_signature" dir="ltr">Andrew Bell<br>
<a href="mailto:andrew.bell.ia@gmail.com" target="_blank">andrew.bell.ia@gmail.com</a></div>
</div>
</body>
</html>