[pdal] Ferry filter not working in LAZ file

Sat Apr 14 07:39:54 PDT 2018

Howard,

Thanks for your suggestions. I updated PDAL to 1.7.1. I didn't notice a
difference with regards to precision, but most* ferry operations work, as
long as they are to existing fields. Because the field I use to store the
DTM elevation (Red) is an integer, I scale it by 100 so there is no loss of
precision.

The pipeline I would like to use is shown below. I transfer the existing
classification to a new dimension, then create a DTM dimension that is
populated in the next step. Because the dimensions are not saved, however,
I use the substitute approach, below, which works. I ferry the
Classification to UserData, which has the same dimension type, then use Red
to store the DTM value x 100 because it is an integer.

Based on a suggestion by Andrew Bell, I tried using the "extra_dims" option
in Writers.las to specify the additional fields created by the ferry filter
and their types, as shown in the first pipeline below. However, the
dimensions are never added to the output file. Are there parameters in
addition to "extra_dims" that I need to specify? VLRs? Do I need to write
to uncompressed LAS files first?

Pete Gadomski's workflow was a great inspiration and helped me develop my
substitute pipeline below. Thank you for the link!

* I noticed some weird behavior with the assign filter that I want to point
out.  When assigning multiple dimensions, sometimes one doesn't get
assigned. It appears to depend on the order of the assignment. For example,
in this stage Classification, Blue and Green are correctly assigned, but
GpsTime is not:

        {
            "type":"filters.assign",
            "assignment":"Classification[:]=0",
            "assignment":"GpsTime[:]=50",
            "assignment":"Blue[:]=10",
            "assignment":"Green[:]=20"
        },

However, when I take out the "GpsTime" assignment and replace it with
"Green", the Green assignment fails, despite having the same value:

        {
            "type":"filters.assign",
            "assignment":"Classification[:]=0",
            "assignment":"Green[:]=20",
            "assignment":"Blue[:]=10"
        },

I got around this by leaving the "GpsTime" spacer in the above example
since my goal was to update Blue and Green only. Apparently the second
"assignment" is being ignored by PDAL.

Thanks,
Eric

*Ideal Pipeline:*

{
  "pipeline":
  [
    "C:/data/forest_park/forest_park_UTM_cropped.laz",
    {
      "type":"filters.ferry",
      "dimensions":"Classification=>OldClassification, =>DTM"
    },
    {
      "type":"filters.colorization",
      "dimensions":"DTM:1:1.0",
      "raster":"C:/data/forest_park/dtm/w001001.adf"
    },
    {
      "type":"writers.las",
      "compression":"true",
      "extra_dims":"OldClassification=uint8, DTM=float",
      "scale_x":"0.01",
      "scale_y":"0.01",
      "scale_z":"0.01",
      "offset_x":"auto",
      "offset_y":"auto",
      "offset_z":"auto",
      "filename":"C:/data/forest_park/forest_park_orig_dtm.laz"
    }
  ]
}

*Substitute Pipeline:*

{
  "pipeline":
  [
    "C:/data/forest_park/forest_park_UTM_cropped.laz",
    {
      "type":"filters.ferry",
      "dimensions":"Classification=>UserData"
    },
    {
      "type":"filters.colorization",
      "dimensions":"Red:1:100.0",
      "raster":"C:/data/forest_park/dtm/w001001.adf"
    },
    {
      "type":"writers.las",
      "compression":"true",
      "scale_x":"0.01",
      "scale_y":"0.01",
      "scale_z":"0.01",
      "offset_x":"auto",
      "offset_y":"auto",
      "offset_z":"auto",
      "filename":"C:/data/forest_park/forest_park_orig_dtm.laz"
    }
  ]
}

On Fri, Apr 13, 2018 at 10:12 AM, Howard Butler <howard at hobu.co> wrote:

>
> On Apr 12, 2018, at 11:05 PM, Eric Robeck <erobeck at gmail.com> wrote:
>
> Hello,
>
> I have written a PDAL pipeline to perform the following steps:
>
> 1.) Ferry the existing Classification to the UserData field for storage
> (this is more efficient than creating a new field)
>
>
> Did you find this to be the case? Adding another dimension shouldn't
> really have an extra cost.
>
> 2.) Use the ferry filter to create a new DTM field
>
>
> 3.) Use the colorize filter to extract the elevation value from a
> single-band TIF (a DTM made from the original ground points)
>
>
> Pete Gadomski made a post about this contorted scenario that's currently
> required. See http://www.gadom.ski/2018/02/28/pdal-hs.html for some
> details.
>
> 4.) Run SMRF to reclassify the ground points
>
> I cannot get the ferry filter to work at all.
>
>
> The syntax in your pipeline is 1.7's syntax, but you stated you're using
> 1.6.0. This might be the discrepancy.  We want to make this scenario
> better. filters.colorization is a bad name for what it does (it's really
> raster overlay). 1.7 fixed the precision issue [1] when doing the task.
>
> It won't transfer values between existing columns or create new columns,
> despite using the same syntax described on PDAL.io. I have tried various
> ways to get around this issue. Changing the order of filters.ferry in the
> pipeline didn't make a difference. I saw that the online examples worked on
> uncompressed LAS files, so I converted the input files from LAZ to LAS but
> there was no change. The pipeline completes without errors, but info
> --stats and info --schema show that the ferried fields were not created or
> updated.
>
> The colorize filter works fine, as long as I use an existing field. For
> example, "dimensions":"Red:1:1000.0" works by mapping the elevation values
> to the Red field and multiplying by 1000 to preserve millimeter precision
> in an integer field. However, "dimensions":"DTM:1:1000.0" does not, because
> the ferry filter doesn't create the DTM field. The SMRF filter also works
> well, but without the ability to ferry values I lose the original
> classifications.
>
> I am using PDAL 1.6.0 on Windows within OSGeo4W. It was installed this
> month. My pipeline is copied below.
>
>
> PDAL 1.7.1 is now available via OSGeo4W64. I don't know that this will
> solve any issues for you, but we will be talking about the same codebase as
> far as functionality goes. I know that the filters.ferry syntax was amended
> to be more explicit about where data from dimensions should be going.
>
> Hope this helps. Pete's write-up is a good one of how to achieve what
> you're after.
>
> [1] https://github.com/PDAL/PDAL/pull/1841
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20180414/8564d645/attachment.html>