[pdal] Python pipeline not reading ept.json

[Howard Butler]

Howard

Ah, ArcGIS conda environments. That's an entirely different bucket of eels. 

I would suggest trying to operate in a clean non-ArcGIS environment that's known to work. We are available for a support contract if being inside ArcGIS is a hard requirement for you, but it's not something that's usually simple to address.

I wish ArcGIS used the principle of "small environments", but unfortunately, their default environment installs the universe and then users are stuck waiting for it to never solve or end up with linking conflicts like these. 

Howard



> On Aug 19, 2022, at 6:11 AM, Gelder, Brian K [A&BE] <bkgelder at iastate.edu> wrote:
> 
> Howard,
> Thanks for your tip; should have thought of that myself. It does appear to be an issue with Python and not pdal directly. I'm guessing it has to do with the fact that I am using ArcGIS Pro 3.0 for my Python 3.9 install. I installed Pro, cloned the default Python install, and then installed via "conda install -c conda-forge pdal" Does anyone have suggestions on how make these two play nicely together?
> 
> I can run pipelines directly from the command line but not through Python. If I run the following pdal command inside my ArcGIS Pro configured Python command prompt "pdal pipeline iowa.json --debug"
> 
> The pipeline then runs successfully. However if I try to run "pdal --drivers" at the same command line I get the following error messages. After acknowledging the errors it then proceeds to print the drivers and I can still run pipelines.
> 
> "pdal.exe - Entry Point No Found"
> 
> "The procedure entry point ?mergeMode at Filter@pdal@@EEBA?AW4WhereMergeMode at State@2 at XZ could not be located in the dynamic link library C:\Users\bkgelder\AppData\Local\ESRI\conda\envs\pda-testing-env\lib\site_packages\bin\libpdal_plugin_filter_numpy.dll"
> 
> "The procedure entry point ?mergeMode at Reader@pdal@@EEBA?AW4WhereMergeMode at State@2 at XZ could not be located in the dynamic link library C:\Users\bkgelder\AppData\Local\ESRI\conda\envs\pda-testing-env\lib\site_packages\bin\libpdal_plugin_reader_numpy.dll"
> 
> Additionally - if I try to import pdal within a .py script file I receive the following warning about a circular import. I do not receive this when I import pdal in the IDLE shell. 
> 
> "Traceback (most recent call last):
>  File "C:\Users\bkgelder\Box\Scripts\pdal.py", line 4, in <module>
>    import pdal
>  File "C:\Users\bkgelder\Box\Scripts\pdal.py", line 16, in <module>
>    pipeline = pdal.Pipeline(json)
> AttributeError: partially initialized module 'pdal' has no attribute 'Pipeline' (most likely due to a circular import)"
> 
> I've also been trying to update pdal to 2.4.3 but not having luck due to listed conflicts. I've tried "conda update pdal" and "conda install -c conda-forge pdal=2.4.3"
> 
> Thank you,
> Brian
> 
> -----Original Message-----
> From: Howard Butler <howard at hobu.co> 
> Sent: Monday, August 15, 2022 2:26 PM
> To: Gelder, Brian K [A&BE] <bkgelder at iastate.edu>
> Cc: pdal at lists.osgeo.org
> Subject: Re: [pdal] Python pipeline not reading ept.json
> 
> 
> 
>> On Aug 15, 2022, at 12:53 PM, Gelder, Brian K [A&BE] <bkgelder at iastate.edu> wrote:
>> 
>> Hello,
>> I’m a pdal newbie and I’m trying to figure out how to successfully read ept.json files. I have pdal 2.4.2 installed via Conda on Python 3.9 on Win 10 and am able to use the pipeline to process local datasets like interesting.las in the tutorials. However, when I try to read EPT web resources, like in https://pdal.io/en/latest/tutorial/iowa-entwine.html, I receive the following error:
>> 
>> Traceback (most recent call last):
>>  File "<pyshell#15>", line 1, in <module>
>>    pipeline2.validate()
>>  File "C:\Users\bkgelder\AppData\Local\ESRI\conda\envs\pdal-testing-env\lib\site-packages\pdal\pipeline.py", line 43, in validate
>>    return self.p.validate()
>>  File "libpdalpython.pyx", line 185, in pdal.libpdalpython.PyPipeline.validate
>> RuntimeError: readers.ept: Could not read from s3-us-west-2.amazonaws.com/usgs-lidar-public/IA_FullState/ept.json
>> 
>> I get similar failure when I tried to query the smaller Hurricane Florence dataset Vishu wrote about last week. I can view the ept.json files in Firefox. Is there a firewall setting I’m missing?
> 
> Brian,
> 
> The first thing to do is to try a pipeline with 'pdal pipeline' and take Python out of it. Another thing to check is pull a new environment with PDAL 2.4.3 and see if that works. There was a big memory leak of the LAZ decompression in 2.4.0-2.4.2 that you won't want if you're doing things in Python.
> 
> Hope this helps,
> 
> Howard
>