This sounds great. I think from a users perspective, Here's some feedback.<br><br>* Perhaps on the lasinfo data checking, the program could output to user if file contains out of spec components. That way before even processing it one has the info to know if data is "in specification for processing".<br>
<br>* For operations working with the data, either las2las or las2txt, as mentioned, provide a flag for user to allow the out of spec points, or discard them. In my case, it seems like once it encountered an out of spec point, the whole rest of the las file was not processed, and the header only showed 7 acutal, vs 5million header count.<br>
<br>* I like the sounds of flexibility on read, but strictness on write. Propagating bad data is not goot, so skipping out of spec points in the writing out a las seems very reasonable. Knowing the data one has from a vendor is out of spec is also good to know, and the recipient of the data should report the out of spec data back to vendor to have them re-do and return proper files.<br>
<br>* Having options to deal with out of spec points is great, because as you said, sometimes a client may accept the data, not scan it or know the details like you guys do, or even check if the file is good. Once the data is exchanged and paid for, they may lose the ability to request them to fix and resend. This being said, as said before, sometimes users get stuck with this bad data and have to press forward with it, so having the option to know about the out of spec points, and how to handle them is a great feature.<br>
<br>Great application! Very fast. I put the las* commands in a simple perl script, and it cranks right through 23 tiles (5K x 5K ft). Scanning, file, eliminating codes and creating bare earth .txt files for GRASS input. the las files are ~150MB in size, and it just blazes through them. <br>
<br>Mark<br><br><br><div class="gmail_quote">On Mon, Apr 28, 2008 at 12:30 AM, Howard Butler <<a href="mailto:hobu.inc@gmail.com">hobu.inc@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
On Apr 27, 2008, at 5:52 PM, Mateusz Loskot wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Howard Butler wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Mateusz,<br>
Mark in Florida has some 1.1 LAS files that have points with Scan<br>
Angle Ranks that are outside the -90/90 range that the specification<br>
says.<br>
</blockquote>
<br>
Hobu,<br>
<br>
Seems, Mark's file does not follow the LAS specification.<br>
Is that possible?<br>
</blockquote>
<br>
Yes, I agree. I think the file is out of spec.<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
If yes, then I think it would be worth to know what's the generator of<br>
such file and provide appropriate warning/comment in libLAS FAQ.<br>
<br>
</blockquote>
<br>
The offending software writes 1.1 files with the following:<br>
Project ID/GUID: '00000000-0000-0000-0000-000000000776'<br>
System Identifier: 'NIIRS10'<br>
Generating Software: 'LIDAR1 tiled'<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
I just commented out the check for now:<br>
<a href="http://liblas.org/browser/trunk/src/laspoint.cpp#L113" target="_blank">http://liblas.org/browser/trunk/src/laspoint.cpp#L113</a><br>
</blockquote>
<br>
OK<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
This begs the question, what do we do for reading invalid input?<br>
</blockquote>
<br>
I would prefer to keep validating it.<br>
</blockquote>
<br>
I don't know that we can keep validating it *where* we validate currently -- in the getter/setter methods of LASPoint.<br>
<br>
Maybe we could have something like<br>
-- ReadNextPoint(bool validate)<br>
-- if (validate) {LASPoint::validate();}<br>
<br>
and have LASPoint::validate toss an exception. This way we can do it on write as well. I'm not sure out_of_range is the best exception to throw either.<br>
<br>
Some things that might be good candidates for validate():<br>
- Invalid time values<br>
- Scan angle rank<br>
- flight edge not being 1 or 0<br>
- invalid classifcation values<br>
- invalid scan direction<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Scan angle doesn't really mean too much, so for this case, following<br>
the spec exactly isn't going to be a deal, especially if there are<br>
other writers out there who aren't following. But for other items,<br>
especially the flags attributes, this is a big deal.<br>
</blockquote>
<br>
Your suggestion sounds reasonable to me. However, I'd like to propose some modifications (below) :-)<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Any ideas on a reasonable tact to take? Liberal in what we read and<br>
strict in what we write?<br>
</blockquote>
<br>
1. The ASPRS LAS format is suppose to be a data exchange format,<br>
so we always must [sic] write in strict mode. IMHO, if LAS writers are very relaxed, then interoperability - a very principle of ASPRS LAS format - will be on paper only.<br>
</blockquote>
<br>
Such is the fallacy of all standards :) I think we should have LASWriter do validate() on write, and we should not allow folks to circumvent this with a flag. It adds a few checks and might cost some raw write performance, but it is just pushing the existing test that lives in the LASPoint methods around so it isn't a net gain or loss in performance.<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
<br>
2. I'd vote for libLAS reading in semi-relaxed mode by default, what actually you are suggesting above, but in strict mode on request (a configuration setting, a program parameter, etc.).<br>
</blockquote>
<br>
For read operations, I think we should have a flag that allows you to use relaxed reading, but we should default to validate() on read as well, IMO. For translation operations like las2las, this might mean throwing out points that aren't to spec, because write() isn't going to allow it. I would have las2las provide summary in that instance.<br>
<br>
The only way to get people to write files in spec are to tell them they're writing it out of spec :) Also, I would venture a guess that libLAS is the first LAS format library out there to really care about validating the parts of the base spec we implement.<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
<br>
3. While reading LAS file in semi-relaxed, the library should always print a warning to stdout.<br>
</blockquote>
<br>
I have changed all of the utilities to emit text output to stderr because stdin/stdout can be used for regular file i/o for some of them. IMO, the library shouldn't print error messages anywhere, and it should notify the application that there's an error or warning and let it decide what to do. I already have stuff in the C API's error handling to be able to deal with this, I think.<br>
<br>
Anyone else with thoughts on this issue, please chime in. I don't want to box folks in because a lot of times you don't control who's giving you LAS files to deal with. On the other hand, I don't want to perpetuate crappy broken data either...<br>
<br>
Howard<br>
_______________________________________________<br>
Liblas-devel mailing list<br>
<a href="mailto:Liblas-devel@mail.hobu.net" target="_blank">Liblas-devel@mail.hobu.net</a><br>
<a href="http://mail.hobu.net/mailman/listinfo/liblas-devel" target="_blank">http://mail.hobu.net/mailman/listinfo/liblas-devel</a><br>
</blockquote></div><br>