[pdal] Python pipeline not reading ept.json

[Gelder, Brian K [A&BE]]

Howard,
Thanks for your tip; should have thought of that myself. It does appear to be an issue with Python and not pdal directly. I'm guessing it has to do with the fact that I am using ArcGIS Pro 3.0 for my Python 3.9 install. I installed Pro, cloned the default Python install, and then installed via "conda install -c conda-forge pdal" Does anyone have suggestions on how make these two play nicely together?

I can run pipelines directly from the command line but not through Python. If I run the following pdal command inside my ArcGIS Pro configured Python command prompt "pdal pipeline iowa.json --debug"

The pipeline then runs successfully. However if I try to run "pdal --drivers" at the same command line I get the following error messages. After acknowledging the errors it then proceeds to print the drivers and I can still run pipelines.

"pdal.exe - Entry Point No Found"

"The procedure entry point ?mergeMode at Filter@pdal@@EEBA?AW4WhereMergeMode at State@2 at XZ could not be located in the dynamic link library C:\Users\bkgelder\AppData\Local\ESRI\conda\envs\pda-testing-env\lib\site_packages\bin\libpdal_plugin_filter_numpy.dll"

"The procedure entry point ?mergeMode at Reader@pdal@@EEBA?AW4WhereMergeMode at State@2 at XZ could not be located in the dynamic link library C:\Users\bkgelder\AppData\Local\ESRI\conda\envs\pda-testing-env\lib\site_packages\bin\libpdal_plugin_reader_numpy.dll"

Additionally - if I try to import pdal within a .py script file I receive the following warning about a circular import. I do not receive this when I import pdal in the IDLE shell. 

"Traceback (most recent call last):
  File "C:\Users\bkgelder\Box\Scripts\pdal.py", line 4, in <module>
    import pdal
  File "C:\Users\bkgelder\Box\Scripts\pdal.py", line 16, in <module>
    pipeline = pdal.Pipeline(json)
AttributeError: partially initialized module 'pdal' has no attribute 'Pipeline' (most likely due to a circular import)"

I've also been trying to update pdal to 2.4.3 but not having luck due to listed conflicts. I've tried "conda update pdal" and "conda install -c conda-forge pdal=2.4.3"

Thank you,
Brian

-----Original Message-----
From: Howard Butler <howard at hobu.co> 
Sent: Monday, August 15, 2022 2:26 PM
To: Gelder, Brian K [A&BE] <bkgelder at iastate.edu>
Cc: pdal at lists.osgeo.org
Subject: Re: [pdal] Python pipeline not reading ept.json



> On Aug 15, 2022, at 12:53 PM, Gelder, Brian K [A&BE] <bkgelder at iastate.edu> wrote:
> 
> Hello,
> I’m a pdal newbie and I’m trying to figure out how to successfully read ept.json files. I have pdal 2.4.2 installed via Conda on Python 3.9 on Win 10 and am able to use the pipeline to process local datasets like interesting.las in the tutorials. However, when I try to read EPT web resources, like in https://pdal.io/en/latest/tutorial/iowa-entwine.html, I receive the following error:
>  
> Traceback (most recent call last):
>   File "<pyshell#15>", line 1, in <module>
>     pipeline2.validate()
>   File "C:\Users\bkgelder\AppData\Local\ESRI\conda\envs\pdal-testing-env\lib\site-packages\pdal\pipeline.py", line 43, in validate
>     return self.p.validate()
>   File "libpdalpython.pyx", line 185, in pdal.libpdalpython.PyPipeline.validate
> RuntimeError: readers.ept: Could not read from s3-us-west-2.amazonaws.com/usgs-lidar-public/IA_FullState/ept.json
>  
> I get similar failure when I tried to query the smaller Hurricane Florence dataset Vishu wrote about last week. I can view the ept.json files in Firefox. Is there a firewall setting I’m missing?

Brian,

The first thing to do is to try a pipeline with 'pdal pipeline' and take Python out of it. Another thing to check is pull a new environment with PDAL 2.4.3 and see if that works. There was a big memory leak of the LAZ decompression in 2.4.0-2.4.2 that you won't want if you're doing things in Python.

Hope this helps,

Howard