[Liblas-devel] libLAS Desires

[Howard Butler]

On Mar 11, 2010, at 2:12 PM, Murphy, Kevin E. wrote:

> We’re considering moving QT Modeler over to using libLAS for its LAS reading/writing needs.  

Excellent.  I think libLAS has a lot to offer over maintaining your own including handling of the geotiff stuff, cross-platform operation, and simple API.  Also, we have a very strong incentive to ensure we're writing as precisely to the specification as we can and reading all the garbage that's handed to us as best we can :)

> The loveliness of LAS1.3 and the potential variability of LAS2.0 has me desiring to get out of maintaining my own LAS parser.

I doubt libLAS is going to do anything with LAS 1.3 waveform data.  It appears to me as though the market is rejecting this format as it only fits one sensor, and if anything, people are just going to store point data in LAS files and call it 1.3.

> QT in some modes accesses LAS files somewhat like a disk-resident database or memory-mapped file so that it can access point-record data at will without keeping it all memory resident.  This requires random access reads (and possibly eventually writes) to/from LAS files.  

libLAS currently takes in std::istream for read and std::ostream for writes.  If the implementation is seekable, like in the case of an std::ifstream for example, then random access to the point data in the form of ReadPointAt() is enabled.   

> Depending upon the nature of the query I am making, this could be one single point, several thousand points relatively contained within a certain section of the LAS files, millions of points spread randomly all across the file, or plain old fashion sequential access to the whole kit-and-kaboodle.

In my own cursory experiments of maneuvering around the file while guided by a spatial index, the performance hit to hop around and read things wasn't that noticeable, but I'm sure there's a cost to it.  

> I don’t really know how libLAS caches access to LAS files internally,

Currently, it doesn't cache anything at at all.  With the new ReaderI interface, it would be possible to provide a cached reader implementation that could live in parallel with the existing serial one.  As to what a point cache's actual implementation looks like, I'm a bit unsure.  What defines the "uniqueness" of a point in a LAS file (ie, how do we know we have a cache hit)?  Just its index position?
 
> It would also be nice if such access were threaded such that I could request one block of points, fire off a thread to read another, and be processing away on the first block while the second request is being filled.  Heck, multiple threads all churning away on their own sub-sections requesting additional points as needed while we’re at it.

I think you could do this now by defining multiple liblas::Readers for each thread and controlling the flow of incoming points yourself (how to request, size of request once GetPointsAt were enabled and cached).  I'm skeptical of having multiple threads touch the same liblas::Reader, as there is a bunch of state information carried along that could easily get mangled.  Others may have more informed opinions...  
 
> More forward-looking, I’d like to have a SetPoint() and SetPoints() to overwrite data records in an LAS.

I would support implementing these.

> No interest on my part in an “InsertPoints” at this time.

I don't think we want to get into the insertion business.  In my mind, if you want a true database, use a true database -- LAS is awful for this :)  

Thanks for the feedback, and I'm excited to have you giving us a strong look.

Howard