[Liblas-devel] Cached Reader bottleneck

[Mateusz Loskot]

On 21/10/10 11:54, Mateusz Loskot wrote:
> On 21/10/10 04:39, Gary Huber wrote:
>> Mat,
>>
>> I got your change and it works to initialize a Reader but on a later
>> Reader::Reset() call it runs out of memory. Should it be creating a
>> new array on Reset?
>
> In theory, it should not, but I'm not sure what is the context of
> resetting the reader. I will have to learn about it.

I forgot about one more thing.
Simply, fetching header like this:

{
	liblas::Reader reader(ifs);
	header = reader.GetHeader();
}

causes creation of complete caching engine, including
allocation of masks array. If this array is very large...

Next, running las2las file.las will cause more than (literally 3 or 4) 
constructions of LASReader object, thus allocation of the cache masks 
array. Long story short, las2las will try to allocate your huge array of
700 millions masks 3 or 4 times! Even if memory is released, subsequent 
allocations may simply fail due to fragmentation.

I think we should extract a lightweight LASHeaderReader from LASReader
for uses cases like the above, to avoid setting up complete reading 
engine if user wants to get header only.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net
Charter Member of OSGeo, http://osgeo.org
Member of ACCU, http://accu.org