[pdal] FW: FW: Attempting to filter out (1) Unclassified points flagged as Withheld
Rose Phillips
RPhillips at linz.govt.nz
Wed Aug 4 15:14:45 PDT 2021
But this was mentioned before by Brad in the thread below?
"Also, the where option is not supported on reader/writer stages, so you actually should've seen an error if you executed the pipeline that you shared. If you simply want a pipeline to extract non-withheld points, you can use the range filter. If you want to skip withheld points, but not discard them, you could use where with the other filters in your pipeline"
Any chance you could give me an example of this pipeline?
From: Andrew Bell <andrew.bell.ia at gmail.com>
Sent: Thursday, 5 August 2021 10:10 a.m.
To: Rose Phillips <RPhillips at linz.govt.nz>
Cc: Bradley Chambers <brad.chambers at gmail.com>; pdal <pdal at lists.osgeo.org>
Subject: Re: [pdal] FW: Attempting to filter out (1) Unclassified points flagged as Withheld
This is incorrect.
'where' selects points that are included in the filter/writer being run.
You're just trying to remove some points from the writer, so all you have to do is use the 'where' clause in the writer. You don't need the range filter if you take this approach.
On Wed, Aug 4, 2021, 6:04 PM Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>> wrote:
Hi Andrew and Brad
Unfortunately I'm very confused, this is my interpretation of the help I've been given so far (thank you for this).
My objective: To filter out withheld flagged points under classification (1) unclassified, for further processing.
This is the pipeline using the same file as below from Open Topography NZ20_Westport dataset (CL2_BR20_2020_1000_4134.laz)
I have created a pipeline incorporating both suggestions made filtering out (1) unclassified points flagged as withheld e.g via bit reference and where ClassFlag statement.
Let me know if this is incorrect.
{
"pipeline" : [
{
"type" : "readers.las",
"filename" : "input.las"
},
{
"type":"filters.range",
"limits":"Classification![129:129]",
"where":"ClassFlags!=4 || Classification != 1"
},
{
"type" : "writers.las",
"a_srs" : "EPSG:2193+7839",
"filesource_id" : "0",
"filename" : "output.las"
}
]
}
I still have these points in my output las file;
(pdal) C:\>pdal info C:\Temp\scan_angle\where_option\pdal_test\processed\CL2_BR20_2020_1000_4134.laz --filters.stats.count "Classification,ClassFlags" --filters.stats.dimensions "Classification,ClassFlags"
(pdal info filters.stats Warning) Dimension 'ClassFlags' listed in --dimensions option does not exist. Ignoring.
(pdal info filters.stats Warning) Dimension 'ClassFlags' listed in --count option does not exist. Ignoring.
{
"file_size": 8079232,
"filename": "C:\\Temp\\scan_angle\\where_option\\pdal_test\\processed\\CL2_BR20_2020_1000_4134.laz",
"now": "2021-08-05T10:00:28+1200",
"pdal_version": "2.3.0 (git-version: Release)",
"reader": "readers.las",
"stats":
{
"statistic":
[
{
"average": 39.47002407,
"count": 2524409,
"counts":
[
"1.000000/27800",
"2.000000/1154344",
"3.000000/160785",
"4.000000/135106",
"5.000000/9930",
"6.000000/304542",
"7.000000/106",
"18.000000/1",
"129.000000/731795"
],
"maximum": 129,
"minimum": 1,
"name": "Classification",
"position": 0,
"stddev": 57.21758565,
"variance": 3273.852107
}
]
}
}
(pdal) C:\>pdal --version
--------------------------------------------------------------------------------
pdal 2.3.0 (git-version: Release)
--------------------------------------------------------------------------------
(pdal) C:\>
Many thanks again (nga mihi nui) Rose
From: Andrew Bell <andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>>
Sent: Thursday, 5 August 2021 2:31 a.m.
To: Bradley Chambers <brad.chambers at gmail.com<mailto:brad.chambers at gmail.com>>
Cc: Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>>; pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
Subject: Re: [pdal] FW: Attempting to filter out (1) Unclassified points flagged as Withheld
Rose,
I think the where clause you'd need if your data is 1.4 is:
where : "ClassFlags != 4 || Classification != 1"
On Wed, Aug 4, 2021 at 10:14 AM Andrew Bell <andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>> wrote:
"where" is supported for writers as of version 2.3
On Wed, Aug 4, 2021 at 10:13 AM Bradley Chambers <brad.chambers at gmail.com<mailto:brad.chambers at gmail.com>> wrote:
Rose,
As Andrew suggested, the withheld bit is actually stored in ClassFlags for your data. In this case, withheld returns are marked as "4" (see the LAS spec for more on the flags).
$ pdal info CL2_BR20_2020_1000_4134.laz --filters.stats.count "Classification,ClassFlags" --filters.stats.dimensions "Classification,ClassFlags"
{
"file_size": 8728570,
"filename": "CL2_BR20_2020_1000_4134.laz",
"now": "2021-08-04T08:51:53-0500",
"pdal_version": "2.2.0 (git-version: 3dd320)",
"reader": "readers.las",
"stats":
{
"statistic":
[
{
"average": 2.3644045,
"count": 2524409,
"counts":
[
"1.000000/759595",
"2.000000/1154344",
"3.000000/160785",
"4.000000/135106",
"5.000000/9930",
"6.000000/304542",
"7.000000/106",
"18.000000/1"
],
"maximum": 18,
"minimum": 1,
"name": "Classification",
"position": 0,
"stddev": 1.55327344,
"variance": 2.412658379
},
{
"average": 1.159550612,
"count": 2524409,
"counts":
[
"0.000000/1792614",
"4.000000/731795"
],
"maximum": 4,
"minimum": 0,
"name": "ClassFlags",
"position": 1,
"stddev": 1.814840525,
"variance": 3.29364613
}
]
}
}
Also, the where option is not supported on reader/writer stages, so you actually should've seen an error if you executed the pipeline that you shared. If you simply want a pipeline to extract non-withheld points, you can use the range filter. If you want to skip withheld points, but not discard them, you could use where with the other filters in your pipeline.
Brad
On Wed, Aug 4, 2021 at 8:12 AM Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>> wrote:
LAZ files with 129 bits are downloadable for free here :-
aws s3 cp s3://pc-bulk/NZ20_Westport/ . --recursive --endpoint-url https://opentopography.s3.sdsc.edu<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopentopography.s3.sdsc.edu%2F&data=04%7C01%7CRPhillips%40linz.govt.nz%7C6e0cec85a17e4288eac308d957949da9%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637637118092738408%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t8vk5DVCeNcu5Ddwq%2BaRjwUdJjzH4vqol0ps39cgeyY%3D&reserved=0> --no-sign-request
Using the same pipeline as below I was having the same issue with example file from the above bucket CL2_BR20_2020_1000_4134.laz - output file still contained the 129 bit when explicitly filtered out by writers.las.where statement.
From: Andrew Bell <andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>>
Sent: Wednesday, 4 August 2021 11:57 a.m.
To: Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>>
Cc: pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
Subject: Re: [pdal] Attempting to filter out (1) Unclassified points flagged as Withheld
The pipeline doesn't seem to match the output you show, so it's hard to know what's going on. If you want to share a file I can see what's up.
On Tue, Aug 3, 2021 at 5:22 PM Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>> wrote:
Hi Andrew,
Thank you for the heads up, I tried the following pipeline;
{
"pipeline" : [
{
"type" : "readers.las",
"filename" : "input.las"
},
{
"type" : "writers.las",
"a_srs" : "EPSG:2193+7839",
"filesource_id" : "0",
"filename" : "output.las",
"where": "ClassFlags!=129"
}
]
}
However my output LAZ file still had these bits intact;
{
"file_size": 3007189,
"filename": "C:\\Temp\\scan_angle\\test_file\\processed\\CL2_BW24_2020_1000_4526.laz",
"now": "2021-08-04T09:20:17+1200",
"pdal_version": "2.3.0 (git-version: Release)",
"reader": "readers.las",
"stats":
{
"statistic":
[
{
"average": 6.704382019,
"count": 1307434,
"maximum": 129,
"minimum": 2,
"name": "Classification",
"position": 0,
"stddev": 21.69585237,
"values":
[
2,
3,
4,
5,
7,
9,
18,
129
],
"variance": 470.7100099
}
]
}
}
Kindest regards Rose
______________________________________________________________________________________________________________________________________________________________
From: Andrew Bell <andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>>
Sent: Tuesday, 3 August 2021 11:00 p.m.
To: Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>>
Cc: pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
Subject: Re: [pdal] Attempting to filter out (1) Unclassified points flagged as Withheld
On Mon, Aug 2, 2021 at 5:39 PM Rose Phillips <RPhillips at linz.govt.nz<mailto:RPhillips at linz.govt.nz>> wrote:
Hi pdal'ers
These are my attempts to filter out (1) Unclassified Points flagged as withheld in LAS filers for further analyses.
Any chance you guys can point to where I have gone wrong?
pipelines I have tried using pdal 2.3.0: to filter out (1) unclassified points flagged as withheld;
1.
{
"pipeline" : [
{
"type" : "readers.las",
"filename" : "input.las"
},
{
"type":"filters.range",
"limits":"Classification![129:129]"
},
{
"type" : "writers.las",
"a_srs" : "EPSG:2193+7839",
"filesource_id" : "0",
"filename" : "output.las"
}
]
}
I think that this should filter out unclassified withheld points with LAS 1.2 and 1.3. With LAS 1.4, the withheld attribute is stored in the "ClassFlags" dimension, so you'll need to use boolean logic with that dimension and the "Classification" dimension to do what you want. You might also use pdal info --enumerate to make sure that your data has the values you think it does.
--
Andrew Bell
andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>
________________________________
This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info at linz.govt.nz<mailto:info at linz.govt.nz>) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.
--
Andrew Bell
andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>
_______________________________________________
pdal mailing list
pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/pdal<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpdal&data=04%7C01%7CRPhillips%40linz.govt.nz%7C6e0cec85a17e4288eac308d957949da9%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637637118092738408%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BUsJ8WfbrEV1b%2B9P8styNjtU04J%2FBC%2FLg0D47%2B9ypDI%3D&reserved=0>
_______________________________________________
pdal mailing list
pdal at lists.osgeo.org<mailto:pdal at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/pdal<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpdal&data=04%7C01%7CRPhillips%40linz.govt.nz%7C6e0cec85a17e4288eac308d957949da9%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637637118092748362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rI%2BWlTlk5i9Uuf%2BSixqZLJI9Cn2Ti5Xbi0JchL9j86s%3D&reserved=0>
--
Andrew Bell
andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>
--
Andrew Bell
andrew.bell.ia at gmail.com<mailto:andrew.bell.ia at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20210804/3e1e2e16/attachment-0001.html>
More information about the pdal
mailing list