[pdal] [EXTERNAL] Re: reader.ept error - Could read from

Pearson, Scott pearsonsm at ornl.gov
Mon Jul 8 07:00:54 PDT 2024 

Hi Michael,

I believe you are correct.  When I switched to a computer not connected to my company’s VPN, the PDAL pipeline executed successfully.

Also, thank you to Howard, Kirk, and Paul for responding to my post. I consider this  problem solved.

Scott

From: michael.smith.erdc at gmail.com <michael.smith.erdc at gmail.com>
Sent: Monday, July 8, 2024 8:49 AM
To: Pearson, Scott <pearsonsm at ornl.gov>
Cc: Howard Butler <howard at hobu.co>; pdal at lists.osgeo.org
Subject: Re: [pdal] [EXTERNAL] Re: reader.ept error - Could read from

This is not uncommon on some corporate networks, they are doing MITM checks and blocking the certificate checks. You might need to see if they have a specific certificate that you should now be using. There is some openssl option to disable the http.schannelCheckRevoke false. You might need to do something curl calls against the s3 url and see what the certificate is that is being used and see if its no longer signed by amazon but by your companies certificate.


Michael Smith
US Army Corps



On Jul 8, 2024, at 3:36 PM, Pearson, Scott via pdal <pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>> wrote:

Hi Howard,
Here’s what I see after setting  VERBOSE=1 :


pdal pipeline Plot_176612216020004.json --debug
(PDAL Debug) Debugging...
(pdal pipeline readers.ept Debug) Curl config:
        timeout: 5s
        followRedirect: true
        verifyPeer: true
        caBundle: (default)
        caInfo: (default)
        Proxy: (default)
* Host s3-us-west-2.amazonaws.com:443 was resolved.
* IPv6: (none)
* IPv4: 52.92.147.120, 52.92.128.104, 52.218.236.96, 52.92.152.216, 52.218.183.0, 52.218.237.32, 52.92.180.72, 52.92.138.0
*   Trying 52.92.147.120:443...
* Connected to s3-us-west-2.amazonaws.com (52.92.147.120) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection
* schannel: shutting down SSL/TLS connection with s3-us-west-2.amazonaws.com port 443
Curl failure: SSL connect error
* Hostname s3-us-west-2.amazonaws.com was found in DNS cache
*   Trying 52.92.147.120:443...
* Connected to s3-us-west-2.amazonaws.com (52.92.147.120) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection
* schannel: shutting down SSL/TLS connection with s3-us-west-2.amazonaws.com port 443
Curl failure: SSL connect error
* Hostname s3-us-west-2.amazonaws.com was found in DNS cache
*   Trying 52.92.147.120:443...
* Connected to s3-us-west-2.amazonaws.com (52.92.147.120) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection
* schannel: shutting down SSL/TLS connection with s3-us-west-2.amazonaws.com port 443
Curl failure: SSL connect error
* Hostname s3-us-west-2.amazonaws.com was found in DNS cache
*   Trying 52.92.147.120:443...
* Connected to s3-us-west-2.amazonaws.com (52.92.147.120) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection
* schannel: shutting down SSL/TLS connection with s3-us-west-2.amazonaws.com port 443
Curl failure: SSL connect error
* Hostname s3-us-west-2.amazonaws.com was found in DNS cache
*   Trying 52.92.147.120:443...
* Connected to s3-us-west-2.amazonaws.com (52.92.147.120) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection
* schannel: shutting down SSL/TLS connection with s3-us-west-2.amazonaws.com port 443
Curl failure: SSL connect error
* Hostname s3-us-west-2.amazonaws.com was found in DNS cache
*   Trying 52.92.147.120:443...
* Connected to s3-us-west-2.amazonaws.com (52.92.147.120) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
* Closing connection
* schannel: shutting down SSL/TLS connection with s3-us-west-2.amazonaws.com port 443
Curl failure: SSL connect error
PDAL: readers.ept: Could not read from s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_27County_blk4_2015_LAS_2018/ept.json

Thanks,
Scott



From: Howard Butler <howard at hobu.co<mailto:howard at hobu.co>>
Sent: Monday, July 8, 2024 8:25 AM
To: Pearson, Scott <pearsonsm at ornl.gov<mailto:pearsonsm at ornl.gov>>
Cc: Kirk Waters - NOAA Federal <kirk.waters at noaa.gov<mailto:kirk.waters at noaa.gov>>; pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
Subject: Re: [pdal] [EXTERNAL] Re: reader.ept error - Could read from

set VERBOSE=1 in your environment before running your pipeline and include the --debug switch in your pdal command to see if it will emit more diagnostic for you.



On Jul 8, 2024, at 7:05 AM, Pearson, Scott via pdal <pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>> wrote:

Hi Kirk,

I have seen the problem consistently since July 1 and again this morning.  I use a PC that is on my employer’s VPN, which might be contributing to the issue.

Scott

From: Kirk Waters - NOAA Federal <kirk.waters at noaa.gov<mailto:kirk.waters at noaa.gov>>
Sent: Monday, July 8, 2024 7:35 AM
To: Pearson, Scott <pearsonsm at ornl.gov<mailto:pearsonsm at ornl.gov>>
Cc: Paul Harwood <runette at gmail.com<mailto:runette at gmail.com>>; pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
Subject: Re: [pdal] [EXTERNAL] Re: reader.ept error - Could read from

I've seen this error message before with other files. When I pulled the ept.json file to verify, it looked fine. Rerunning the command also worked fine. This has made me believe it's been a network issue. For my automated processes, I now try to catch that response, sleep a minute, and then try again. Has it been consistently failing?

Kirk Waters, PhD
NOAA Office for Coastal Management
Applied Sciences Program
coast.noaa.gov/digitalcoast<https://urldefense.us/v2/url?u=http-3A__coast.noaa.gov_digitalcoast&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=_j6WrNWoVwy77DWC5EJvBc9LKUL5lV6AlicsKukZeWnLAijORUp4Y6Y6O2QTeSFs&s=py4GkxTULW8v1KQXRdcuOrApgAtGP2DNvXXiG6LFkWU&e=>




On Mon, Jul 8, 2024 at 7:26 AM Pearson, Scott via pdal <pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>> wrote:
Hi Paul,

The URL is quoted in my input JSON file.  In the email message, Microsoft Outlook converted the URL text to a link and dropped the quotes.  Here’s a snippet from the JSON with quotation marks preserved:
{
    "tag": "ept_reader",
    "type": "readers.ept",
    "bounds": "([ -9423306.45485711 , -9421847.59871605 ],[ 4175665.92739632 , 4177131.3660114 ])",
    "filename": “https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_27County_blk4_2015_LAS_2018/ept.json<https://urldefense.us/v2/url?u=https-3A__s3-2Dus-2Dwest-2D2.amazonaws.com_usgs-2Dlidar-2Dpublic_USGS-5FLPC-5FTN-5F27County-5Fblk4-5F2015-5FLAS-5F2018_ept.json&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=_j6WrNWoVwy77DWC5EJvBc9LKUL5lV6AlicsKukZeWnLAijORUp4Y6Y6O2QTeSFs&s=2fzUXpUwgQYrH-rkW6_bGLG3pgVPaUTGCCioQXewQRU&e=>”,
    "resolution": "0.01",
    "threads": "4"
  }

Thanks,
Scott


From: Paul Harwood <runette at gmail.com<mailto:runette at gmail.com>>
Sent: Saturday, July 6, 2024 5:10 AM
To: Pearson, Scott <pearsonsm at ornl.gov<mailto:pearsonsm at ornl.gov>>
Cc: pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
Subject: [EXTERNAL] Re: [pdal] reader.ept error - Could read from

I don't know if this is just an artefact of the email - but :

https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_27County_blk4_2015_LAS_2018/ept.json<https://urldefense.us/v2/url?u=https-3A__s3-2Dus-2Dwest-2D2.amazonaws.com_usgs-2Dlidar-2Dpublic_USGS-5FLPC-5FTN-5F27County-5Fblk4-5F2015-5FLAS-5F2018_ept.json&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=OdmAeyhN4J3b_xFt4WhzHNuQCyNq824pbGFV3UroI6V5tWNz16wjdyIBWpc7MUa1&s=DppyiaHKZxe-lcRF1sMm-tzX3QvrM6XUlmHN-QZEGYo&e=>

should be:

"https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_27County_blk4_2015_LAS_2018/ept.json<https://urldefense.us/v2/url?u=https-3A__s3-2Dus-2Dwest-2D2.amazonaws.com_usgs-2Dlidar-2Dpublic_USGS-5FLPC-5FTN-5F27County-5Fblk4-5F2015-5FLAS-5F2018_ept.json&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=OdmAeyhN4J3b_xFt4WhzHNuQCyNq824pbGFV3UroI6V5tWNz16wjdyIBWpc7MUa1&s=DppyiaHKZxe-lcRF1sMm-tzX3QvrM6XUlmHN-QZEGYo&e=>"

On Fri, 5 Jul 2024 at 17:24, Pearson, Scott via pdal <pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>> wrote:
Dear PDAL folks,
 I have an error using readers.ept in a PDAL pipeline.  Using instructions on the Quickstart page<https://urldefense.us/v2/url?u=https-3A__pdal.io_en_2.7.2_quickstart.html&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=OdmAeyhN4J3b_xFt4WhzHNuQCyNq824pbGFV3UroI6V5tWNz16wjdyIBWpc7MUa1&s=su9lNnhWcUnIS-p1ASVmGNvlVbDN_kimfMgeqAjZdLE&e=>, PDAL (v. 2.7.2) is installed using miniconda (conda v. 24.5.0) and applied updates were applied to conda and pdal installations.  The “pdal info autzen.laz -p 0” command works, so I’ve assume that the PDAL install was successful.
 When I run “pdal pipeline Plot_176612216020004.json –debug”, I receive this output and error:
“(PDAL Debug) Debugging...
(pdal pipeline readers.ept Debug) PDAL: readers.ept: Could not read from s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_27County_blk4_2015_LAS_2018/ept.json<https://urldefense.us/v2/url?u=http-3A__s3-2Dus-2Dwest-2D2.amazonaws.com_usgs-2Dlidar-2Dpublic_USGS-5FLPC-5FTN-5F27County-5Fblk4-5F2015-5FLAS-5F2018_ept.json&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=OdmAeyhN4J3b_xFt4WhzHNuQCyNq824pbGFV3UroI6V5tWNz16wjdyIBWpc7MUa1&s=grZ4fqpuKLbRQaHxZRQsSUZEWar0gYK3B0ovPUvWjCc&e=> “
 The contents of Plot_176612216020004.json are:
[
  {
    "tag": "ept_reader",
    "type": "readers.ept",
    "bounds": "([ -9423306.45485711 , -9421847.59871605 ],[ 4175665.92739632 , 4177131.3660114 ])",
    "filename": “https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_27County_blk4_2015_LAS_2018/ept.json<https://urldefense.us/v2/url?u=https-3A__s3-2Dus-2Dwest-2D2.amazonaws.com_usgs-2Dlidar-2Dpublic_USGS-5FLPC-5FTN-5F27County-5Fblk4-5F2015-5FLAS-5F2018_ept.json&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=_j6WrNWoVwy77DWC5EJvBc9LKUL5lV6AlicsKukZeWnLAijORUp4Y6Y6O2QTeSFs&s=2fzUXpUwgQYrH-rkW6_bGLG3pgVPaUTGCCioQXewQRU&e=>”,
    "resolution": "0.01",
    "threads": "4"
  },
  {
    "tag": "las_writer",
    "type": "writers.las",
    "filename": "USGS_LPC_TN_27County_blk4_2015_LAS_2018_176612216020004.laz",
    "compression": "laszip",
    "extra_dims": "all",
    "forward": "header, vlr",
    "offset_x": "auto",
    "offset_y": "auto",
    "offset_z": "auto",
    "scale_x": "0.01",
    "scale_y": "0.01",
    "scale_z": "0.01"
  }
]

I can access the etp.json file listed under filename using my browser and also using a curl command from the conda prompt.
I have also tried the Iowa example from this tutorial and receive the same “Could not read from ….” error.
 Any suggestions for solving this problem?  My overall goal is to download lidar point clouds for a set of 200 field plots (3 ha each).
 Thank you,
Scott
_______________________________________________
pdal mailing list
pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/pdal<https://urldefense.us/v2/url?u=https-3A__lists.osgeo.org_mailman_listinfo_pdal&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=OdmAeyhN4J3b_xFt4WhzHNuQCyNq824pbGFV3UroI6V5tWNz16wjdyIBWpc7MUa1&s=PST3C28-L5iw9E-wtUJA0LshiHNBg78HGTpOn1OH4aU&e=>
_______________________________________________
pdal mailing list
pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/pdal<https://urldefense.us/v2/url?u=https-3A__lists.osgeo.org_mailman_listinfo_pdal&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=_j6WrNWoVwy77DWC5EJvBc9LKUL5lV6AlicsKukZeWnLAijORUp4Y6Y6O2QTeSFs&s=EsfMSNspm-xBh9dAu7TmVnuGSiMpcPqQkW6N1vKcGsc&e=>
_______________________________________________
pdal mailing list
pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/pdal<https://urldefense.us/v2/url?u=https-3A__lists.osgeo.org_mailman_listinfo_pdal&d=DwQFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=NZShMMfOuJLHoVeR7aB7hrcmvywgrppC_teCbyKze5ubAwUlnvaCqttmDCBeE1YK&s=hgUgsHFzkUKKI49JHAS1pCUA-Dwiwa87WVP2JSGfejE&e=>

_______________________________________________
pdal mailing list
pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/pdal<https://urldefense.us/v2/url?u=https-3A__lists.osgeo.org_mailman_listinfo_pdal&d=DwQFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=HORY163nBbFgyWK0c3Xwo0vnkU1QghnN6V30DBX36lU&m=x3LGGYV3kc3Ol0jeI0lYu6yhQGZMORvNmn7M8Errrov4DnHo5_jGNPO8-yG-8QYJ&s=o-A2w3mMYtX3wAzTIV7i3L9ndLyq1Tb_aNg1uUN3Lzg&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20240708/7c337d5c/attachment-0001.htm>

More information about the pdal mailing list