[pdal] FW: Attempting to filter out (1) Unclassified points flagged as Withheld
Andrew Bell
andrew.bell.ia at gmail.com
Wed Aug 4 15:09:38 PDT 2021
This is incorrect.
'where' selects points that are included in the filter/writer being run.
You're just trying to remove some points from the writer, so all you have
to do is use the 'where' clause in the writer. You don't need the range
filter if you take this approach.
On Wed, Aug 4, 2021, 6:04 PM Rose Phillips <RPhillips at linz.govt.nz> wrote:
> Hi Andrew and Brad
>
>
>
> Unfortunately I’m very confused, this is my interpretation of the help
> I’ve been given so far (thank you for this).
>
>
>
> My objective: To filter out withheld flagged points under classification
> (1) unclassified, for further processing.
>
>
>
> This is the pipeline using the same file as below from Open Topography
> NZ20_Westport dataset (CL2_BR20_2020_1000_4134.laz)
>
>
> I have created a pipeline incorporating both suggestions made filtering
> out (1) unclassified points flagged as withheld e.g via bit reference and
> where ClassFlag statement.
>
> Let me know if this is incorrect.
>
>
>
> {
>
> "pipeline" : [
>
> {
>
> "type" : "readers.las",
>
> "filename" : "input.las"
>
> },
>
> {
>
> "type":"filters.range",
>
> "limits":"Classification![129:129]",
>
> "where":"ClassFlags!=4 || Classification != 1"
>
> },
>
> {
>
> "type" : "writers.las",
>
> "a_srs" : "EPSG:2193+7839",
>
> "filesource_id" : "0",
>
> "filename" : "output.las"
>
> }
>
> ]
>
> }
>
>
>
>
>
> I still have these points in my output las file;
>
>
>
> (pdal) C:\>pdal info
> C:\Temp\scan_angle\where_option\pdal_test\processed\CL2_BR20_2020_1000_4134.laz
> --filters.stats.count "Classification,ClassFlags"
> --filters.stats.dimensions "Classification,ClassFlags"
>
> (pdal info filters.stats Warning) Dimension 'ClassFlags' listed in
> --dimensions option does not exist. Ignoring.
>
> (pdal info filters.stats Warning) Dimension 'ClassFlags' listed in --count
> option does not exist. Ignoring.
>
> {
>
> "file_size": 8079232,
>
> "filename":
> "C:\\Temp\\scan_angle\\where_option\\pdal_test\\processed\\CL2_BR20_2020_1000_4134.laz",
>
> "now": "2021-08-05T10:00:28+1200",
>
> "pdal_version": "2.3.0 (git-version: Release)",
>
> "reader": "readers.las",
>
> "stats":
>
> {
>
> "statistic":
>
> [
>
> {
>
> "average": 39.47002407,
>
> "count": 2524409,
>
> "counts":
>
> [
>
> "1.000000/27800",
>
> "2.000000/1154344",
>
> "3.000000/160785",
>
> "4.000000/135106",
>
> "5.000000/9930",
>
> "6.000000/304542",
>
> "7.000000/106",
>
> "18.000000/1",
>
> "129.000000/731795"
>
> ],
>
> "maximum": 129,
>
> "minimum": 1,
>
> "name": "Classification",
>
> "position": 0,
>
> "stddev": 57.21758565,
>
> "variance": 3273.852107
>
> }
>
> ]
>
> }
>
> }
>
>
>
> (pdal) C:\>pdal --version
>
>
> --------------------------------------------------------------------------------
>
> pdal 2.3.0 (git-version: Release)
>
>
> --------------------------------------------------------------------------------
>
>
>
>
>
> (pdal) C:\>
>
>
>
> Many thanks again (nga mihi nui) Rose
>
>
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Thursday, 5 August 2021 2:31 a.m.
> *To:* Bradley Chambers <brad.chambers at gmail.com>
> *Cc:* Rose Phillips <RPhillips at linz.govt.nz>; pdal at lists.osgeo.org
> *Subject:* Re: [pdal] FW: Attempting to filter out (1) Unclassified
> points flagged as Withheld
>
>
>
> Rose,
>
> I think the where clause you'd need if your data is 1.4 is:
>
> where : "ClassFlags != 4 || Classification != 1"
>
>
>
> On Wed, Aug 4, 2021 at 10:14 AM Andrew Bell <andrew.bell.ia at gmail.com>
> wrote:
>
> "where" is supported for writers as of version 2.3
>
>
>
> On Wed, Aug 4, 2021 at 10:13 AM Bradley Chambers <brad.chambers at gmail.com>
> wrote:
>
> Rose,
>
>
>
> As Andrew suggested, the withheld bit is actually stored in ClassFlags for
> your data. In this case, withheld returns are marked as "4" (see the LAS
> spec for more on the flags).
>
>
>
> $ pdal info CL2_BR20_2020_1000_4134.laz --filters.stats.count
> "Classification,ClassFlags" --filters.stats.dimensions
> "Classification,ClassFlags"
> {
> "file_size": 8728570,
> "filename": "CL2_BR20_2020_1000_4134.laz",
> "now": "2021-08-04T08:51:53-0500",
> "pdal_version": "2.2.0 (git-version: 3dd320)",
> "reader": "readers.las",
> "stats":
> {
> "statistic":
> [
> {
> "average": 2.3644045,
> "count": 2524409,
> "counts":
> [
> "1.000000/759595",
> "2.000000/1154344",
> "3.000000/160785",
> "4.000000/135106",
> "5.000000/9930",
> "6.000000/304542",
> "7.000000/106",
> "18.000000/1"
> ],
> "maximum": 18,
> "minimum": 1,
> "name": "Classification",
> "position": 0,
> "stddev": 1.55327344,
> "variance": 2.412658379
> },
> {
> "average": 1.159550612,
> "count": 2524409,
> "counts":
> [
> "0.000000/1792614",
> "4.000000/731795"
> ],
> "maximum": 4,
> "minimum": 0,
> "name": "ClassFlags",
> "position": 1,
> "stddev": 1.814840525,
> "variance": 3.29364613
> }
> ]
> }
> }
>
>
>
> Also, the where option is not supported on reader/writer stages, so you
> actually should've seen an error if you executed the pipeline that you
> shared. If you simply want a pipeline to extract non-withheld points, you
> can use the range filter. If you want to skip withheld points, but not
> discard them, you could use where with the other filters in your pipeline.
>
>
>
> Brad
>
>
>
> On Wed, Aug 4, 2021 at 8:12 AM Rose Phillips <RPhillips at linz.govt.nz>
> wrote:
>
> LAZ files with 129 bits are downloadable for free here :-
>
> aws s3 cp s3://pc-bulk/NZ20_Westport/ . --recursive --endpoint-url
> https://opentopography.s3.sdsc.edu
> <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopentopography.s3.sdsc.edu%2F&data=04%7C01%7CRPhillips%40linz.govt.nz%7C9dc3b69a0d2642f3e48b08d957549791%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637636843122216352%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZpSbGUy1yu1VS3FJX1XNa35xOtyPRYre2gBy6hkqYgg%3D&reserved=0>
> --no-sign-request
>
>
>
> Using the same pipeline as below I was having the same issue with example
> file from the above bucket CL2_BR20_2020_1000_4134.laz – output file still
> contained the 129 bit when explicitly filtered out by writers.las.where
> statement.
>
>
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Wednesday, 4 August 2021 11:57 a.m.
> *To:* Rose Phillips <RPhillips at linz.govt.nz>
> *Cc:* pdal at lists.osgeo.org
> *Subject:* Re: [pdal] Attempting to filter out (1) Unclassified points
> flagged as Withheld
>
>
>
> The pipeline doesn't seem to match the output you show, so it's hard to
> know what's going on. If you want to share a file I can see what's up.
>
>
>
> On Tue, Aug 3, 2021 at 5:22 PM Rose Phillips <RPhillips at linz.govt.nz>
> wrote:
>
> Hi Andrew,
>
>
>
> Thank you for the heads up, I tried the following pipeline;
>
>
>
> {
>
> "pipeline" : [
>
> {
>
> "type" : "readers.las",
>
> "filename" : "input.las"
>
> },
>
> {
>
> "type" : "writers.las",
>
> "a_srs" : "EPSG:2193+7839",
>
> "filesource_id" : "0",
>
> "filename" : "output.las",
>
> "where": "ClassFlags!=129"
>
> }
>
> ]
>
> }
>
>
>
> However my output LAZ file still had these bits intact;
>
>
>
> {
>
> "file_size": 3007189,
>
> "filename": "C:\\Temp\\scan_angle\\test_file\\processed\\
> CL2_BW24_2020_1000_4526.laz",
>
> "now": "2021-08-04T09:20:17+1200",
>
> "pdal_version": "2.3.0 (git-version: Release)",
>
> "reader": "readers.las",
>
> "stats":
>
> {
>
> "statistic":
>
> [
>
> {
>
> "average": 6.704382019,
>
> "count": 1307434,
>
> "maximum": 129,
>
> "minimum": 2,
>
> "name": "Classification",
>
> "position": 0,
>
> "stddev": 21.69585237,
>
> "values":
>
> [
>
> 2,
>
> 3,
>
> 4,
>
> 5,
>
> 7,
>
> 9,
>
> 18,
>
> 129
>
> ],
>
> "variance": 470.7100099
>
> }
>
> ]
>
> }
>
> }
>
>
>
> Kindest regards Rose
>
>
> *______________________________________________________________________________________________________________________________________________________________*
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Tuesday, 3 August 2021 11:00 p.m.
> *To:* Rose Phillips <RPhillips at linz.govt.nz>
> *Cc:* pdal at lists.osgeo.org
> *Subject:* Re: [pdal] Attempting to filter out (1) Unclassified points
> flagged as Withheld
>
>
>
>
>
>
>
> On Mon, Aug 2, 2021 at 5:39 PM Rose Phillips <RPhillips at linz.govt.nz>
> wrote:
>
> Hi pdal’ers
>
>
>
> These are my attempts to filter out (1) Unclassified Points flagged as
> withheld in LAS filers for further analyses.
>
> Any chance you guys can point to where I have gone wrong?
>
>
>
> pipelines I have tried using pdal 2.3.0: to filter out (1) unclassified
> points flagged as withheld;
>
>
>
> 1.
>
>
>
> {
>
> "pipeline" : [
>
> {
>
> "type" : "readers.las",
>
> "filename" : "input.las"
>
> },
>
> {
>
> "type":"filters.range",
>
> "limits":"Classification![129:129]"
>
> },
>
> {
>
> "type" : "writers.las",
>
> "a_srs" : "EPSG:2193+7839",
>
> "filesource_id" : "0",
>
> "filename" : "output.las"
>
> }
>
> ]
>
> }
>
>
>
> I think that this should filter out unclassified withheld points with LAS
> 1.2 and 1.3. With LAS 1.4, the withheld attribute is stored in the
> "ClassFlags" dimension, so you'll need to use boolean logic with that
> dimension and the "Classification" dimension to do what you want. You might
> also use pdal info --enumerate to make sure that your data has the values
> you think it does.
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
>
> ------------------------------
>
>
> This message contains information, which may be in confidence and may be
> subject to legal privilege. If you are not the intended recipient, you must
> not peruse, use, disseminate, distribute or copy this message. If you have
> received this message in error, please notify us immediately (Phone 0800
> 665 463 or info at linz.govt.nz) and destroy the original message. LINZ
> accepts no responsibility for changes to this email, or for any
> attachments, after its transmission from LINZ. Thank You.
>
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
> <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpdal&data=04%7C01%7CRPhillips%40linz.govt.nz%7C9dc3b69a0d2642f3e48b08d957549791%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637636843122216352%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4Z5WxzWs0VPrQNUu0OJzYfmwNf061DST2C5o9xHV0go%3D&reserved=0>
>
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
> <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpdal&data=04%7C01%7CRPhillips%40linz.govt.nz%7C9dc3b69a0d2642f3e48b08d957549791%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637636843122226308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9jjxz8zSNfp5GvPSEMBCf3yCOfZVUE9BeFLzzS87Rfk%3D&reserved=0>
>
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20210804/6eb7e9c4/attachment-0001.html>
More information about the pdal
mailing list