[Liblas-devel] Throwing out_of_range exceptions on read

[M S]

This sounds great.  I think from a users perspective, Here's some feedback.

*  Perhaps on the lasinfo data checking, the program could output to user if
file contains out of spec components.  That way before even processing it
one has the info to know if data is "in specification for processing".

*  For operations working with the data, either las2las or las2txt, as
mentioned, provide a flag for user to allow the out of spec points, or
discard them.   In my case, it seems like once it encountered an out of spec
point, the whole rest of the las file was not processed, and the header only
showed 7 acutal, vs 5million header count.

*  I like the sounds of flexibility on read, but strictness on write.
Propagating bad data is not goot, so skipping out of spec points in the
writing out a las seems very reasonable.  Knowing the data one has from a
vendor is out of spec is also good to know, and the recipient of the data
should report the out of spec data back to vendor to have them re-do and
return proper files.

*  Having options to deal with out of spec points is great, because as you
said, sometimes a client may accept the data, not scan it or know the
details like you guys do, or even check if the file is good.  Once the data
is exchanged and paid for, they may lose the ability to request them to fix
and resend.  This being said, as said before, sometimes users get stuck with
this bad data and have to press forward with it, so having the option to
know about the out of spec points, and how to handle them is a great
feature.

Great application!  Very fast.  I put the las* commands in a simple perl
script, and it cranks right through 23 tiles (5K x 5K ft).  Scanning, file,
eliminating codes and creating bare earth .txt files for GRASS input.  the
las files are ~150MB in size, and it just blazes through them.

Mark


On Mon, Apr 28, 2008 at 12:30 AM, Howard Butler <hobu.inc at gmail.com> wrote:

>
> On Apr 27, 2008, at 5:52 PM, Mateusz Loskot wrote:
>
>  Howard Butler wrote:
> >
> > > Mateusz,
> > > Mark in Florida has some 1.1 LAS files that have points with Scan
> > > Angle Ranks that are outside the -90/90 range that the specification
> > > says.
> > >
> >
> > Hobu,
> >
> > Seems, Mark's file does not follow the LAS specification.
> > Is that possible?
> >
>
> Yes, I agree.  I think the file is out of spec.
>
> >
> > If yes, then I think it would be worth to know what's the generator of
> > such file and provide appropriate warning/comment in libLAS FAQ.
> >
> >
> The offending software writes 1.1 files with the following:
>  Project ID/GUID:           '00000000-0000-0000-0000-000000000776'
>  System Identifier:         'NIIRS10'
>  Generating Software:       'LIDAR1 tiled'
>
>
>  I just commented out the check for now:
> > > http://liblas.org/browser/trunk/src/laspoint.cpp#L113
> > >
> >
> > OK
> >
> >  This begs the question, what do we do for reading invalid input?
> > >
> >
> > I would prefer to keep validating it.
> >
>
> I don't know that we can keep validating it *where* we validate currently
> -- in the getter/setter methods of LASPoint.
>
> Maybe we could have something like
> -- ReadNextPoint(bool validate)
> -- if (validate) {LASPoint::validate();}
>
> and have LASPoint::validate toss an exception.  This way we can do it on
> write as well.  I'm not sure out_of_range is the best exception to throw
> either.
>
> Some things that might be good candidates for validate():
>  - Invalid time values
>  - Scan angle rank
>  - flight edge not being 1 or 0
>  - invalid classifcation values
>  - invalid scan direction
>
>
>
> >
> >  Scan angle doesn't really mean too much, so for this case, following
> > > the spec exactly isn't going to be a deal, especially if there are
> > > other writers out there who aren't following. But for other items,
> > > especially the flags attributes, this is a big deal.
> > >
> >
> > Your suggestion sounds reasonable to me. However, I'd like to propose
> > some modifications (below) :-)
> >
> >  Any ideas on a reasonable tact to take?  Liberal in what we read and
> > > strict in what we write?
> > >
> >
> > 1. The ASPRS LAS format is suppose to be a data exchange format,
> > so we always must [sic] write in strict mode. IMHO, if LAS writers are
> > very  relaxed, then interoperability - a very principle of ASPRS LAS format
> > - will be on paper only.
> >
>
> Such is the fallacy of all standards :)  I think we should have LASWriter
> do validate() on write, and we should not allow folks to circumvent this
> with a flag.  It adds a few checks and might cost some raw write
> performance, but it is just pushing the existing test that lives in the
> LASPoint methods around so it isn't a net gain or loss in performance.
>
>
> >
> > 2. I'd vote for libLAS reading in semi-relaxed mode by default, what
> > actually you are suggesting above, but in strict mode on request (a
> > configuration setting, a program parameter, etc.).
> >
>
> For read operations, I think we should have a flag that allows you to use
> relaxed reading, but we should default to validate() on read as well, IMO.
>  For translation operations like las2las, this might mean throwing out
> points that aren't to spec, because write() isn't going to allow it.  I
> would have las2las provide summary in that instance.
>
> The only way to get people to write files in spec are to tell them they're
> writing it out of spec :)  Also, I would venture a guess that libLAS is the
> first LAS format library out there to really care about validating the parts
> of the base spec we implement.
>
>
> >
> > 3. While reading LAS file in semi-relaxed, the library should always
> > print a warning to stdout.
> >
>
> I have changed all of the utilities to emit text output to stderr because
> stdin/stdout can be used for regular file i/o for some of them.  IMO, the
> library shouldn't print error messages anywhere, and it should notify the
> application that there's an error or warning and let it decide what to do.
>  I already have stuff in the C API's error handling to be able to deal with
> this, I think.
>
> Anyone else with thoughts on this issue, please chime in.  I don't want to
> box folks in because a lot of times you don't control who's giving you LAS
> files to deal with.  On the other hand, I don't want to perpetuate crappy
> broken data either...
>
> Howard
> _______________________________________________
> Liblas-devel mailing list
> Liblas-devel at mail.hobu.net
> http://mail.hobu.net/mailman/listinfo/liblas-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/liblas-devel/attachments/20080428/0eef3cf7/attachment.html