[pdal] limit reading EPT and writing laz?

[Kirk Waters - NOAA Federal]

Thanks. I wouldn't have guessed it would chew through the memory on a
machine with 80Gb of RAM, but I see it's already at 18Gb after a few
minutes and still climbing. Thanks for the tip on the threads.

Kirk


On Thu, Jan 6, 2022 at 9:39 AM Andrew Bell <andrew.bell.ia at gmail.com> wrote:

> On Thu, Jan 6, 2022 at 9:02 AM Kirk Waters - NOAA Federal <
> kirk.waters at noaa.gov> wrote:
>
>> Andrew,
>> Thanks for testing that. I ran it again with the same result, but this
>> time turned on --developer-debug. Here's what I ran:
>> (gdal) ocm-s-ares$ pdal pipeline --stdin --developer-debug < pipeline.txt
>> > output.txt 2>&1
>> Killed
>> (gdal) ocm-s-ares$ echo $?
>> 137
>> (gdal) ocm-s-ares$ ls -l extractept0.laz
>> -rw-rw-r-- 1 kirk.waters crs 2784906231 Jan  6 07:49 extractept0.laz
>>
>> The output.txt file contained the same thing I remember seeing before,
>> with nothing extra from the debug:
>> (pdal pipeline readers.ept Warning) 2598817459 will be downloaded
>> (pdal pipeline writers.las Warning) Auto offset for X requested in stream
>> mode.  Using value of -80.8347.
>> (pdal pipeline writers.las Warning) Auto offset for Y requested in stream
>> mode.  Using value of 35.3408.
>> (pdal pipeline writers.las Warning) Auto offset for Z requested in stream
>> mode.  Using value of 246.85.
>>
>> All of this is running on RHEL7 in a conda environment with pdal version
>> 2.3.0 (git-version 0800a2). The "Killed" on the terminal seems odd since
>> anything going to stdout or stderr should have landed in output.txt, but
>> that didn't. Almost like something external killed it.
>>
>
> I'm guessing you ran out of memory. That's almost always what it means
> when your process is "killed" by the operating system. This can happen if
> you are able to download data faster than you can consume it. You might try
> setting the number of threads *lower*, as this will reduce the number of
> outstanding requests for data. I have made an adjustment in the copc reader
> for this issue, but it hasn't made it into the ept reader at this time. On
> *nix, run top and watch the memory for your process.
>
> Hope that helps,
>
> --
> Andrew Bell
> andrew.bell.ia at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20220106/3ca3fb5b/attachment.html>