[pdal] FW: FW: Attempting to filter out (1) Unclassified points flagged as Withheld
Andrew Bell
andrew.bell.ia at gmail.com
Thu Aug 5 06:58:08 PDT 2021
[
"input.las",
{
"type":"writers.las",
"filename":"out.las",
"where": "ClassFlags!=4 || Classification != 1"
}
]
On Wed, Aug 4, 2021 at 6:14 PM Rose Phillips <RPhillips at linz.govt.nz> wrote:
> But this was mentioned before by Brad in the thread below?
>
>
>
> “Also, the where option is not supported on reader/writer stages, so you
> actually should've seen an error if you executed the pipeline that you
> shared. If you simply want a pipeline to extract non-withheld points, you
> can use the range filter. If you want to skip withheld points, but not
> discard them, you could use where with the other filters in your pipeline”
>
>
>
> Any chance you could give me an example of this pipeline?
>
>
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Thursday, 5 August 2021 10:10 a.m.
> *To:* Rose Phillips <RPhillips at linz.govt.nz>
> *Cc:* Bradley Chambers <brad.chambers at gmail.com>; pdal <
> pdal at lists.osgeo.org>
> *Subject:* Re: [pdal] FW: Attempting to filter out (1) Unclassified
> points flagged as Withheld
>
>
>
> This is incorrect.
>
>
>
> 'where' selects points that are included in the filter/writer being run.
>
>
>
> You're just trying to remove some points from the writer, so all you have
> to do is use the 'where' clause in the writer. You don't need the range
> filter if you take this approach.
>
>
>
> On Wed, Aug 4, 2021, 6:04 PM Rose Phillips <RPhillips at linz.govt.nz> wrote:
>
> Hi Andrew and Brad
>
>
>
> Unfortunately I’m very confused, this is my interpretation of the help
> I’ve been given so far (thank you for this).
>
>
>
> My objective: To filter out withheld flagged points under classification
> (1) unclassified, for further processing.
>
>
>
> This is the pipeline using the same file as below from Open Topography
> NZ20_Westport dataset (CL2_BR20_2020_1000_4134.laz)
>
>
> I have created a pipeline incorporating both suggestions made filtering
> out (1) unclassified points flagged as withheld e.g via bit reference and
> where ClassFlag statement.
>
> Let me know if this is incorrect.
>
>
>
> {
>
> "pipeline" : [
>
> {
>
> "type" : "readers.las",
>
> "filename" : "input.las"
>
> },
>
> {
>
> "type":"filters.range",
>
> "limits":"Classification![129:129]",
>
> "where":"ClassFlags!=4 || Classification != 1"
>
> },
>
> {
>
> "type" : "writers.las",
>
> "a_srs" : "EPSG:2193+7839",
>
> "filesource_id" : "0",
>
> "filename" : "output.las"
>
> }
>
> ]
>
> }
>
>
>
>
>
> I still have these points in my output las file;
>
>
>
> (pdal) C:\>pdal info
> C:\Temp\scan_angle\where_option\pdal_test\processed\CL2_BR20_2020_1000_4134.laz
> --filters.stats.count "Classification,ClassFlags"
> --filters.stats.dimensions "Classification,ClassFlags"
>
> (pdal info filters.stats Warning) Dimension 'ClassFlags' listed in
> --dimensions option does not exist. Ignoring.
>
> (pdal info filters.stats Warning) Dimension 'ClassFlags' listed in --count
> option does not exist. Ignoring.
>
> {
>
> "file_size": 8079232,
>
> "filename":
> "C:\\Temp\\scan_angle\\where_option\\pdal_test\\processed\\CL2_BR20_2020_1000_4134.laz",
>
> "now": "2021-08-05T10:00:28+1200",
>
> "pdal_version": "2.3.0 (git-version: Release)",
>
> "reader": "readers.las",
>
> "stats":
>
> {
>
> "statistic":
>
> [
>
> {
>
> "average": 39.47002407,
>
> "count": 2524409,
>
> "counts":
>
> [
>
> "1.000000/27800",
>
> "2.000000/1154344",
>
> "3.000000/160785",
>
> "4.000000/135106",
>
> "5.000000/9930",
>
> "6.000000/304542",
>
> "7.000000/106",
>
> "18.000000/1",
>
> "129.000000/731795"
>
> ],
>
> "maximum": 129,
>
> "minimum": 1,
>
> "name": "Classification",
>
> "position": 0,
>
> "stddev": 57.21758565,
>
> "variance": 3273.852107
>
> }
>
> ]
>
> }
>
> }
>
>
>
> (pdal) C:\>pdal --version
>
>
> --------------------------------------------------------------------------------
>
> pdal 2.3.0 (git-version: Release)
>
>
> --------------------------------------------------------------------------------
>
>
>
>
>
> (pdal) C:\>
>
>
>
> Many thanks again (nga mihi nui) Rose
>
>
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Thursday, 5 August 2021 2:31 a.m.
> *To:* Bradley Chambers <brad.chambers at gmail.com>
> *Cc:* Rose Phillips <RPhillips at linz.govt.nz>; pdal at lists.osgeo.org
> *Subject:* Re: [pdal] FW: Attempting to filter out (1) Unclassified
> points flagged as Withheld
>
>
>
> Rose,
>
> I think the where clause you'd need if your data is 1.4 is:
>
> where : "ClassFlags != 4 || Classification != 1"
>
>
>
> On Wed, Aug 4, 2021 at 10:14 AM Andrew Bell <andrew.bell.ia at gmail.com>
> wrote:
>
> "where" is supported for writers as of version 2.3
>
>
>
> On Wed, Aug 4, 2021 at 10:13 AM Bradley Chambers <brad.chambers at gmail.com>
> wrote:
>
> Rose,
>
>
>
> As Andrew suggested, the withheld bit is actually stored in ClassFlags for
> your data. In this case, withheld returns are marked as "4" (see the LAS
> spec for more on the flags).
>
>
>
> $ pdal info CL2_BR20_2020_1000_4134.laz --filters.stats.count
> "Classification,ClassFlags" --filters.stats.dimensions
> "Classification,ClassFlags"
> {
> "file_size": 8728570,
> "filename": "CL2_BR20_2020_1000_4134.laz",
> "now": "2021-08-04T08:51:53-0500",
> "pdal_version": "2.2.0 (git-version: 3dd320)",
> "reader": "readers.las",
> "stats":
> {
> "statistic":
> [
> {
> "average": 2.3644045,
> "count": 2524409,
> "counts":
> [
> "1.000000/759595",
> "2.000000/1154344",
> "3.000000/160785",
> "4.000000/135106",
> "5.000000/9930",
> "6.000000/304542",
> "7.000000/106",
> "18.000000/1"
> ],
> "maximum": 18,
> "minimum": 1,
> "name": "Classification",
> "position": 0,
> "stddev": 1.55327344,
> "variance": 2.412658379
> },
> {
> "average": 1.159550612,
> "count": 2524409,
> "counts":
> [
> "0.000000/1792614",
> "4.000000/731795"
> ],
> "maximum": 4,
> "minimum": 0,
> "name": "ClassFlags",
> "position": 1,
> "stddev": 1.814840525,
> "variance": 3.29364613
> }
> ]
> }
> }
>
>
>
> Also, the where option is not supported on reader/writer stages, so you
> actually should've seen an error if you executed the pipeline that you
> shared. If you simply want a pipeline to extract non-withheld points, you
> can use the range filter. If you want to skip withheld points, but not
> discard them, you could use where with the other filters in your pipeline.
>
>
>
> Brad
>
>
>
> On Wed, Aug 4, 2021 at 8:12 AM Rose Phillips <RPhillips at linz.govt.nz>
> wrote:
>
> LAZ files with 129 bits are downloadable for free here :-
>
> aws s3 cp s3://pc-bulk/NZ20_Westport/ . --recursive --endpoint-url
> https://opentopography.s3.sdsc.edu
> <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopentopography.s3.sdsc.edu%2F&data=04%7C01%7CRPhillips%40linz.govt.nz%7C6e0cec85a17e4288eac308d957949da9%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637637118092738408%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t8vk5DVCeNcu5Ddwq%2BaRjwUdJjzH4vqol0ps39cgeyY%3D&reserved=0>
> --no-sign-request
>
>
>
> Using the same pipeline as below I was having the same issue with example
> file from the above bucket CL2_BR20_2020_1000_4134.laz – output file still
> contained the 129 bit when explicitly filtered out by writers.las.where
> statement.
>
>
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Wednesday, 4 August 2021 11:57 a.m.
> *To:* Rose Phillips <RPhillips at linz.govt.nz>
> *Cc:* pdal at lists.osgeo.org
> *Subject:* Re: [pdal] Attempting to filter out (1) Unclassified points
> flagged as Withheld
>
>
>
> The pipeline doesn't seem to match the output you show, so it's hard to
> know what's going on. If you want to share a file I can see what's up.
>
>
>
> On Tue, Aug 3, 2021 at 5:22 PM Rose Phillips <RPhillips at linz.govt.nz>
> wrote:
>
> Hi Andrew,
>
>
>
> Thank you for the heads up, I tried the following pipeline;
>
>
>
> {
>
> "pipeline" : [
>
> {
>
> "type" : "readers.las",
>
> "filename" : "input.las"
>
> },
>
> {
>
> "type" : "writers.las",
>
> "a_srs" : "EPSG:2193+7839",
>
> "filesource_id" : "0",
>
> "filename" : "output.las",
>
> "where": "ClassFlags!=129"
>
> }
>
> ]
>
> }
>
>
>
> However my output LAZ file still had these bits intact;
>
>
>
> {
>
> "file_size": 3007189,
>
> "filename": "C:\\Temp\\scan_angle\\test_file\\processed\\
> CL2_BW24_2020_1000_4526.laz",
>
> "now": "2021-08-04T09:20:17+1200",
>
> "pdal_version": "2.3.0 (git-version: Release)",
>
> "reader": "readers.las",
>
> "stats":
>
> {
>
> "statistic":
>
> [
>
> {
>
> "average": 6.704382019,
>
> "count": 1307434,
>
> "maximum": 129,
>
> "minimum": 2,
>
> "name": "Classification",
>
> "position": 0,
>
> "stddev": 21.69585237,
>
> "values":
>
> [
>
> 2,
>
> 3,
>
> 4,
>
> 5,
>
> 7,
>
> 9,
>
> 18,
>
> 129
>
> ],
>
> "variance": 470.7100099
>
> }
>
> ]
>
> }
>
> }
>
>
>
> Kindest regards Rose
>
>
> *______________________________________________________________________________________________________________________________________________________________*
>
> *From:* Andrew Bell <andrew.bell.ia at gmail.com>
> *Sent:* Tuesday, 3 August 2021 11:00 p.m.
> *To:* Rose Phillips <RPhillips at linz.govt.nz>
> *Cc:* pdal at lists.osgeo.org
> *Subject:* Re: [pdal] Attempting to filter out (1) Unclassified points
> flagged as Withheld
>
>
>
>
>
>
>
> On Mon, Aug 2, 2021 at 5:39 PM Rose Phillips <RPhillips at linz.govt.nz>
> wrote:
>
> Hi pdal’ers
>
>
>
> These are my attempts to filter out (1) Unclassified Points flagged as
> withheld in LAS filers for further analyses.
>
> Any chance you guys can point to where I have gone wrong?
>
>
>
> pipelines I have tried using pdal 2.3.0: to filter out (1) unclassified
> points flagged as withheld;
>
>
>
> 1.
>
>
>
> {
>
> "pipeline" : [
>
> {
>
> "type" : "readers.las",
>
> "filename" : "input.las"
>
> },
>
> {
>
> "type":"filters.range",
>
> "limits":"Classification![129:129]"
>
> },
>
> {
>
> "type" : "writers.las",
>
> "a_srs" : "EPSG:2193+7839",
>
> "filesource_id" : "0",
>
> "filename" : "output.las"
>
> }
>
> ]
>
> }
>
>
>
> I think that this should filter out unclassified withheld points with LAS
> 1.2 and 1.3. With LAS 1.4, the withheld attribute is stored in the
> "ClassFlags" dimension, so you'll need to use boolean logic with that
> dimension and the "Classification" dimension to do what you want. You might
> also use pdal info --enumerate to make sure that your data has the values
> you think it does.
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
>
> ------------------------------
>
>
> This message contains information, which may be in confidence and may be
> subject to legal privilege. If you are not the intended recipient, you must
> not peruse, use, disseminate, distribute or copy this message. If you have
> received this message in error, please notify us immediately (Phone 0800
> 665 463 or info at linz.govt.nz) and destroy the original message. LINZ
> accepts no responsibility for changes to this email, or for any
> attachments, after its transmission from LINZ. Thank You.
>
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
> <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpdal&data=04%7C01%7CRPhillips%40linz.govt.nz%7C6e0cec85a17e4288eac308d957949da9%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637637118092738408%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BUsJ8WfbrEV1b%2B9P8styNjtU04J%2FBC%2FLg0D47%2B9ypDI%3D&reserved=0>
>
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
> <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fpdal&data=04%7C01%7CRPhillips%40linz.govt.nz%7C6e0cec85a17e4288eac308d957949da9%7C2134e9617e384c34a22b10da5466b725%7C0%7C0%7C637637118092748362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rI%2BWlTlk5i9Uuf%2BSixqZLJI9Cn2Ti5Xbi0JchL9j86s%3D&reserved=0>
>
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
>
>
>
> --
>
> Andrew Bell
> andrew.bell.ia at gmail.com
>
>
--
Andrew Bell
andrew.bell.ia at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20210805/e1a22bbb/attachment-0001.html>
More information about the pdal
mailing list