[Liblas-devel] Re: las2txt question
Angley, William [USA]
angley_william at bah.com
Wed Nov 24 20:07:48 EST 2010
Hi Team,
Looking at this conversation, I have a few points in mind:
* I'm a fan of reversible transformations. Why make las2txt input unusable for getting back to a .las file when it is only slightly tougher to make it usable for getting back to a .las file? I think the only thing we'd need to add to the output of las2txt to achieve this is a line indicating the arguments passed to the '--parse' option of las2txt.
* If we're open to changing the las2txt header output, why don't we just use the lasinfo header output? I can't see a good reason for doing them differently.
* A very easy way to make lasinfo/las2txt header output parseable is to make it standard `YAML <www.yaml.org>`_ as there are permissively licensed open-source libraries for reading and writing YAML in pretty much all languages. The output that's being written by lasinfo now is very close to YAML as it is... it would take only minor changes to bring it around to that. Unlike XML, YAML is readily readable by untrained humans.
* On the flip side, being able to transform .las data into GML or KML formats would be extremely helpful at times. Among other things, this would allow for the display of LiDAR data in Google Earth and via MapServer.
what this could mean for liblas
===============================
I think we should go for it on all fronts: change lasinfo to output YAML data, change las2txt to output its parse options (and a lasinfo YAML header), and create utilities that can output GML and KML. If we do this, we'll have the ability to work with LiDAR returns in 'six' formats:
* .las (1.0, 1.1, 1.2, ~1.3)
* ESRI Shapefiles
* tab-separated values with YAML headers
* GML
* KML
* Oracle Spatial
It may be more feasible than it sounds; other projects (think GDAL/OGR and GhostScript) support dozens of formats each. If we go the same route, we might want to move from having separate programs for each format option we support to having a single program that accepts options to specify input and output formats. (It's worth noting that ogr2ogr and gs do the exact same thing. Ghostscript also supplies shortcut programs for the most common conversions, like ps2pdf.)
las2txt output as a file format
===============================
Also, anyone who writes a program that depends on the output of las2txt has to think of its output as a file format, and we can make las2txt more useful for them by treating the design of its output as a file format design issue. The combination of simple, open standards that I've suggested (YAML + tab-separated values) is inspired in part by Eric S. Raymond's `The Art of Unix Programming <http://www.faqs.org/docs/artu/>`_.
las2txt output is a lot larger than standard .las files when uncompressed, but compressed las2txt output is about the same size as compressed .las files. Compressed las2txt output is much smaller than uncompressed .las files.
::
-rw-rw-r--+ 1 wmangley3 lidar 214572097 Sep 9 2009 example.las
-rw-rw-r--+ 1 wmangley3 lidar 62547608 Nov 24 16:10 example.las.bz2
-rw-rw-r--+ 1 wmangley3 lidar 448464763 Nov 24 16:28 example.txt
-rw-rw-r--+ 1 wmangley3 lidar 103790665 Nov 24 19:33 example.txt.zip
-rw-rw-r--+ 1 wmangley3 lidar 75914928 Nov 24 16:28 example.txt.bz2
Since las2txt output is a file format, it should have a name, and an extension for those of us whose systems require them. I'd propose the name and extension .LYT for the uncompressed version. This proposed extension is an acronym, from *LiDAR, YAML + Text* or *LiDAR, YAML + Tab-Separated*; `according to fileext.com <http://filext.com/file-extension/LYT>`_, it is used by a few programs but none that are related to LiDAR or geospatial work.
And since the output benefits a lot from compression, I think we should treat a compressed version as a first-rate format. If we went with Info-ZIP compression, we could use .LYZ as our extension; if we went with bzip2, .LYTB.
Giving las2txt output a name would also make it easier to refer to in a GDAL/OGR style driver program.
I don't know how well it would work in general use, but it seems worthwhile to note that word processing file formats have been evolving in this direction for some time. Both OpenDocument Format and Office Open XML are compressed, structured text successors to binary file formats.
Thanks,
-- Will
| William Angley
| Booz Allen Hamilton
| angley_william at bah.com
This message is reStructuredText. Feel free to run docutils on it if you'd like it to look prettier.
-----Original Message-----
From: liblas-devel-bounces at lists.osgeo.org [mailto:liblas-devel-bounces at lists.osgeo.org] On Behalf Of Etienne Bellemare Racine
Sent: Wednesday, November 24, 2010 4:38 PM
To: Howard Butler
Cc: Liblas-devel at lists.osgeo.org
Subject: Re: [Liblas-devel] Re: las2txt question
> Etienne,
>
> I am in the process of rewriting las2txt to use the C++ API (so we get the new-style filters, etc).
Great !
> I have everything done except for the header writing portion. After looking at it a bit, I'm wondering if it is that useful in its current form. Do you expect that las2txt output of a header should be easily parseable, or you expect it to act much like a comment block at the front of a source code?
I'd say it should be easily readable with other tools like a csv
reader/editor. That's the reason I personnaly use the las2txt (or
recommend it to others). For instance load points in PostGIS, or even
load points in a GIS (when the las2shp is unreadable -e.g. in ArcGIS).
> As it is right now, las2txt's header output does neither very well. It would be quite simple to have las2txt output libLAS' xml-ification of the header if the user specified a --header argument in the invocation. Would this be useful?
In my case, I'd say no. It is not usefull because the people that need
txt file are not going to use an xml parser. I'd say the the design
should be targetting something simple at least by default. Output one
file, maybe that option could be useful, for other purposes, but I find
it quite complex for the name of the tool or the use cases I have in mind.
> Or is the intention that the header in the front of a .txt xyz LAS file is must cursory information that can't actually be used to get back to the original LAS file?
I'm not sure I understand well your question, but I think I see the
las2txt as an end. I use it because I need a file I can read with a text
reader. If I want to manipulate the file to create a new las file, I
should do it using with appropriate tools like python for instance.
Hope it helps you.
Cheers for the rewriting,
Etienne
> Howard
>
>
> On Oct 28, 2010, at 3:44 PM, Etienne Bellemare wrote:
>
>> Good point, I'd like to suggest also the ability to specify the header separated the same way the fields are going to be. It will create a much more consistent file. e.g. for the moment you can't use tab to seperate header even if your fields are. Having the ability to set a name for the header could be a nice feature too.
>>
>> Thanks again,
>> Etienne
>>
>> On Thu, Oct 28, 2010 at 4:22 PM, Howard Butler<hobu.inc at gmail.com> wrote:
>>
>> On Oct 28, 2010, at 3:16 PM, Randy Bucciarelli wrote:
>>
>>> Hi Howard,
>>>
>>> Is there a way to format resulting precision of values from the output of las2txt?
>>>
>>> Or does 'las2txt' simply output the precision of the values how there were initially stored when creating LAS files?
>> There isn't, but I agree there should be an option to set/override the precision of x/y/z values (independently) I have rewritten lasinfo and las2las using the new C++ API, and I think las2txt should also be rewritten as well. In summary output and other venues in those rewritten utilities, I have tried to respect the specified scale/precision of the dimension if it was set in the header.
>>
>> Rewriting las2txt similarly to lasinfo and las2las is in my queue, but I probably won't get to it for a bit.
>>
>> Howard
>>
>> PS, this question should be asked on the list, and I have forwarded it there._______________________________________________
>> Liblas-devel mailing list
>> Liblas-devel at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/liblas-devel
>>
>> _______________________________________________
>> Liblas-devel mailing list
>> Liblas-devel at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/liblas-devel
_______________________________________________
Liblas-devel mailing list
Liblas-devel at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/liblas-devel
More information about the Liblas-devel
mailing list