[Liblas-devel] libLAS file size limit

[Howard Butler]

On Jan 16, 2011, at 6:22 PM, Volker Wichmann wrote:

> Hi,
> 
> I'm trying to process a 5.3GB LAS file generated from ERDAS eATE (209.271.771 points, version 1.2, generating software 'ERDAS LASTerrainDataset').

If I recall correctly, eATE might use libLAS to write files so maybe we're all in the same ecosystem at least.

> 
> I'm using liblas 1.2.1 build on windows 7 64bit with MSVC 9.0.
> 
> Processing the file causes no problems as long as I'm using the reader.ReadNextPoint() method. But once I try random access via the reader.ReadPointAt() method, I get segmentation faults.
> 
> We are still sticking to liblas 1.2.1 until 1.6 stable is released. I've tried the OSGeo build of liblas 1.6.0b4, but running e.g. lasinfo on the file returns:
> 
> error: The number of points in the header that was set by the software 'ERDAS LASTerrainDataset' does not match the actual number of points in the file as determined by subtracting the data offset (329) from the file length (1146099079) and dividing by the point record length (26). It also does not perfectly contain an exact number of point data and we cannot infer a point count. Calculated number of points: 44080721 Header-specified number of points: 209271771 Point data remainder: 4

I suspect the problem is fact that MSVC 9's seek method in the standard iostreams doesn't support seeking past the end of 32bits.  It has been cataloged at http://trac.liblas.org/ticket/147 The solution is either use Boost's iostreams (ie, pass that into libLAS which will accommodate windows and seek properly), or upgrade to MSVC 10 ( http://trac.liblas.org/ticket/147#comment:18 ).

The reason these seeks still exist is the check that attempts to infer the point count.  As was discussed in another thread a couple of months ago, this should probably go away, and we should just believe the header.  Doing this would solve the problem too.  I will try to visit this for the next beta release this week.

Does a linux or OSX build complain about the file with the same error?  If so, then the above commentary is wrong and it might actually be something with the file, but considering you can sequentially read through it, I doubt it.

> PS: The file has RGB values coded in 8bit (0-255), I thought the LAS standard is to rescale RGB to 16bit?


Yes the (1.3, but not 1.2) standard says to scale it to 16bit, but there are many softwares that don't.  From a practical standpoint, you need to come up with some heuristics to determine what you might be dealing with.