[pdal] 'pdal info' VLR error troubleshooting

[Howard Butler]

> On Apr 27, 2023, at 11:49 AM, Speed, Cole M <cole.speed at beg.utexas.edu> wrote:
> 
> Hello all,
> 
> I am using 'pdal info' to view the metadata for a few laz files. For several of the files, pdal info returns the error: 
> 
> PDAL: readers.las: VLR 1(SF/0) size too large -- flows into point data.
> 
> I believe this may indicate that the header/VLR s longer than is expected, causing the metadata information to be misinterpreted. Attempting to troubleshoot the issue by having a look at the lasreader source code (https://github.com/PDAL/PDAL/blob/master/io/LasReader.cpp), I see the below code snippet:
>         if ((uint64_t)stream->tellg() + vlr.promisedDataSize > d->header.pointOffset)
>             throwError("VLR " + std::to_string(i + 1) +
>                 "(" + vlr.userId + "/" + std::to_string(vlr.recordId) + ") "
>                 "size too large -- flows into point data.");
> 
> I interpret this to indicate an underlying issue related to the laz header/VLR, but I am unsure what specifically this means or how to resolve it. The files are also unreadable by QGIS (Error: 'readers.las: Invalid VLR - exceeds specified file range'). Insight from anyone who has encountered these errors or has any thoughts as to what specifically they imply about the header/VLR formatting of these laz files would be greatly appreciated. Thanks in advance!

I cycled this issue with Cole and a couple things are going on...

1) His pipeline was writing a LAS 1.2 file, which is the default minor_version for writers.las
2) His input data had scan_angle values that were too big for LAS 1.2 even though it says it is a 1.2 file
3) PDAL was throwing an error when it encountered a scan angle that was too large and trying to write it in the output in stream mode
4) The resulting file was partially written and not flushed.

His workaround is to use filters.assign to strip off the offending scan angles and everything works as expected.

I'm not sure there's a fix for us though. We can't guarantee a properly written file in stream mode in this scenario. If you were to write data in --nostream mode, you would get a valid but partial file, but that takes a lot of memory and is slow for pipelines like this. 

Any ideas on how things should behave from anyone?

Howard