[pdal] obtaining classifications from a .ept file

[Connor Manning]

The `pdal info --summary` command simply summarizes the metadata from
https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_MD_VA_Sandy_NCR_2014_LAS_2015/ept.json,
so it won't let you get the classification values.

On the other hand, the `pdal info --stats` command will read all of the
data into memory, which will attempt to create a 30-billion point buffer
for this dataset.  This giant buffer could potentially be skipped if
streaming capability were added to the EPT reader, but you'd still have to
read the 30-billion points in this scenario, so even with compression you'd
be looking at a ~100GB download.

What you might consider is not reading *all* of the data.  The EPT reader
in 1.9 (currently on master, but not included in 1.8) will support a
`resolution` option.  See doc here:
https://github.com/PDAL/PDAL/blob/master/doc/stages/readers.ept.rst

You could then use your current `translate` command and add to it
`--readers.ept.resolution=20` to read the data only to approximately a 20
meter resolution (for example).  While this is not a bulletproof approach
since all of the data is not read, with some experimentation of the
resolution you could potentially get a list of the contained
classifications with reasonable confidence.  Keep in mind that the lack of
EPT streaming means that the points will still be read into an ever-growing
buffer, so you may still run into memory limitations with too high of a
resolution.

Here is a sample output for the dataset you asked about to a 20 meter
resolution with 12 threads.  For me, it took about 1.5 minutes and used
about 2.5 GB of RAM:

$ time pdal info --stats --filters.stats.dimensions=Classification
--filters.stats.count=Classification ept://
https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_MD_VA_Sandy_NCR_2014_LAS_2015
--readers.ept.resolution=20 --readers.ept.threads=12
{
  "filename": "ept://
https://s3-us-west-2.amazonaws.com/usgs-lidar-public/USGS_LPC_MD_VA_Sandy_NCR_2014_LAS_2015
",
  "pdal_version": "1.8.0 (git-version: 755ac1)",
  "stats":
  {
    "statistic":
    [
      {
        "average": 5.63230439,
        "count": 54747896,
        "counts":
        [
          "1.000000/24956352",
          "2.000000/13821592",
          "7.000000/102236",
          "9.000000/2314365",
          "10.000000/15019",
          "11.000000/30058",
          "17.000000/10022141",
          "18.000000/3399744",
          "25.000000/86389"
        ],
        "maximum": 25,
        "minimum": 1,
        "name": "Classification",
        "position": 0,
        "stddev": 6.882052297,
        "variance": 47.36264381
      }
    ]
  }
}

real 1m24.623s
user 1m24.079s
sys 0m4.859s


Hopefully this is helpful.

- Connor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20190206/5e75a16b/attachment.html>