[pdal] PDAL pipeline issues with large datasets

[Nandigam, Viswanath]

Hi Howard,
It looks like an issue with other datasets as well. This works fine in the 2.4.1 version of PDAL but fails with 2.4.2.
For example this pipeline file also has the same issue:

{
    "pipeline": [{
        "type": "readers.ept",
        "filename": "https://s3-us-west-2.amazonaws.com/usgs-lidar-public/NC_HurricaneFlorence_4_2020/ept.json",
        "bounds": "([-8726998.51984671, -8723414.496106366], [4353500.799531042,4356350.524642287])"
    },
    {
        "type":"filters.reprojection",
        "in_srs":"EPSG:3857",
        "out_srs":"EPSG:32617"
    },{
        "type" : "writers.las",
        "filename": "points.laz",
        "a_srs": "EPSG:32617"
    }]}


But a smaller area works fine with PDAL 2.4.2:

{
    "pipeline": [{
        "type": "readers.ept",
        "filename": "https://s3-us-west-2.amazonaws.com/usgs-lidar-public/NC_HurricaneFlorence_4_2020/ept.json",
        "bounds": "([-8726177.25654457, -8725958.025231255], [4354975.9833829915,4355134.666378456])"
    },
    {
        "type":"filters.reprojection",
        "in_srs":"EPSG:3857",
        "out_srs":"EPSG:32617"
    },{
        "type" : "writers.las",
        "filename": "points.laz",
        "a_srs": "EPSG:32617"
    }]}

- Vishu



> On Aug 5, 2022, at 1:34 PM, Howard Butler <howard at hobu.co> wrote:
> 
> 
> 
>> On Aug 5, 2022, at 3:33 PM, Nandigam, Viswanath <vnandigam at ucsd.edu> wrote:
>> 
>> Hi All,
>> I’m trying to get pdal pipeline working with the latest install of pdal via conda. It works for very small areas but is erroring out on larger selections.
>> 
>> The pipeline.json file is
>> 
>> {
>>  "pipeline": [{
>>    "type": "readers.ept",
>>    "filename": "https://urldefense.com/v3/__https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_TN_Middle_B2_2018_LAS_2019/ept.json__;!!Mih3wA!C2u_-GeGjgNG7WYj-fL-7YiN55JN5s1fZEfp9-22Vq_zIzMwsYv5lr7ZlNk0tC77Bfthm_OiC_uB0pom$ ",
>>    "bounds": "([-9666713.882998705, -9662911.140841518], [4283077.817410117,4287281.853965801])"
>>  },
>>  {
>>    "type":"filters.reprojection",
>>    "in_srs":"EPSG:3857",
>>    "out_srs":"EPSG:32616"
>>  },{
>>    "type" : "writers.las",
>>    "filename": "points.laz",
>>    "a_srs": "EPSG:32616"
>>  }]}
>> 
>> 
>> The version of pdal (via conda) is
>> 
>> pdal --version
>> -----------------------------------------------------
>> pdal 2.4.2 (git-version: e4772e)
>> -----------------------------------------------------
>> 
>> I have tried it on two machines with both Ubuntu 22.04 LTS and Ubuntu 18.04.
>> It seems to work with PDAL version 2.4.1.
>> 
>> I see the following error:
>>> PDAL: readers.las: VLR 1(SF/0) size too large -- flows into point data.
>> 
>> Maybe I’m missing something. Is anyone else having this issue? Can anyone please advice? 
>> Thank you!
> 
> I think the issue is one of the LAZ tiles in that EPT is bad.
>