[Liblas-devel] Re: las2txt question

[Howard Butler]

On Nov 24, 2010, at 7:07 PM, Angley, William [USA] wrote:

> Hi Team,
> 
> Looking at this conversation, I have a few points in mind:
> 
> *       I'm a fan of reversible transformations.  Why make las2txt input unusable for getting back to a .las file when it is only slightly tougher to make it usable for getting back to a .las file?  

text is a last resort interchange format for LiDAR data, and I don't think it was ever intended to be anything other than that. Defining a text format that is transferrable back into LAS is a different proposition than letting people define their own text formats like we already do with the --parse option.  I think our intent with las2txt is not to make a transitive format, but rather allow people to use it as the interchange of last resort for feeding data into their own systems.  

> I think the only thing we'd need to add to the output of las2txt to achieve this is a line indicating the arguments passed to the '--parse' option of las2txt.

... and agree on the meaning of what the parse arguments mean.  ... and agree on what their data types are.  ... and so on.  

> *       If we're open to changing the las2txt header output, why don't we just use the lasinfo header output?  I can't see a good reason for doing them differently.

I have been working on a reStructured text output format for lasinfo and friends as well as the XML and custom text formats that are already available.  There are already a number of parsers in a number of languages for ReST.  lasinfo's existing default format is an homage to Martin's original format with some additions and a bit prettier formatting.  

Mostly my question was to Etienne about he would want for more flexibility in the header output because he was asking about this earlier.

> *       A very easy way to make lasinfo/las2txt header output parseable is to make it standard `YAML <www.yaml.org>`_ as there are permissively licensed open-source libraries for reading and writing YAML in pretty much all languages.   The output that's being written by lasinfo now is very close to YAML as it is... it would take only minor changes to bring it around to that.  Unlike XML, YAML is readily readable by untrained humans.

Yes, that's why I was going for reStructured text.  I'm familiar with YAML, but I much prefer ReST :)

> *       On the flip side, being able to transform .las data into GML or KML formats would be extremely helpful at times.  

/me imagines dropping 100 million points into a GML file and handing it off to GeoServer.  The practical implications of LiDAR data often trump the format transformation issues. Something else has to happen to LiDAR data before pushing it into existing GIS systems.  Martin's done some awesome stuff with DEM generation and KML, for example.  

> Among other things, this would allow for the display of LiDAR data in Google Earth and via MapServer.

By "display of LiDAR data," this most likely means elevation models and such -- not point cloud data directly.

> 
> what this could mean for liblas
> ===============================
> 
> I think we should go for it on all fronts: change lasinfo to output YAML data, change las2txt to output its parse options (and a lasinfo YAML header), and create utilities that can output GML and KML.  If we do this, we'll have the ability to work with LiDAR returns in 'six' formats:
> 
> *       .las (1.0, 1.1, 1.2, ~1.3)
> *       ESRI Shapefiles
> *       tab-separated values with YAML headers
> *       GML
> *       KML
> *       Oracle Spatial
> 
> It may be more feasible than it sounds; other projects (think GDAL/OGR and GhostScript) support dozens of formats each.  If we go the same route, we might want to move from having separate programs for each format option we support to having a single program that accepts options to specify input and output formats.  (It's worth noting that ogr2ogr and gs do the exact same thing.  Ghostscript also supplies shortcut programs for the most common conversions, like ps2pdf.)

I think this is beyond the scope of libLAS -- though I agree for the need for a separate GDAL-like library for the constellation of point cloud data formats.  Any text-based format, YAML or otherwise, is not the pivot point for those transformations, however, IMO.  

In my opinion, the quest for the Ultimate LiDAR Format (I hereby trademark ULF haha!) is clearly as much of a fool's errand as the quest for an ultimate raster or vector format.  There's always going to be space for a number of point cloud formats that are going to do some things better than others as well as a need to translate to and from them. 

> 
> las2txt output as a file format
> ===============================
> 
> Also, anyone who writes a program that depends on the output of las2txt has to think of its output as a file format, and we can make las2txt more useful for them by treating the design of its output as a file format design issue.    

In theory, I think this is true, but from a practical standpoint, there is already tons of code and processes depending on las2txt and the custom formats they've already "written" with it to feed data into their softwares.  We can't abandon these folks by changing the requirements and expectations of las2txt.  We can, however, set new requirements in green-field software development areas like libPC, which is expected to be my next major software development effort once we finally get the libLAS 1.6 final release out the door.

Thanks for the feedback and thoughts to help feel out where the boundaries of libLAS are and where those of other software development efforts like libPC might be.

Howard