[Liblas-devel] Appending to files with LASWriter

[Howard Butler]

On Feb 22, 2008, at 5:02 AM, Mateusz Loskot wrote:

> Howard Butler wrote:
>> On Feb 21, 2008, at 3:27 PM, Howard Butler wrote:
>>> Thanks Mateusz.  Now to figure out why writing into an existing
>>> file with LASFile is not working...
>> If I were to run my test program over and over again (we use  
>> std::ios::ate), should I expect that the point count will continually
>> increase?
>
> This is a problem.
> In terms of the implementation of WriterImpl::UpdateHeader
> it won't increase.
>
> 1. Writer does not read data, it writes. So, it does not know anything
> about the header existing in a file being updated.
> 2. The program sets false/invalid number of points (1000 but actually
> writes 1) and writer believes you here
> 3. Writer is able to track number of actually written points in single
> run, and correct the false number initially given in the header
> 4. Writer is not able to track number of points across program  
> execution
> and verify if the number given in header is correct or not.
>
> Also, according to my sense of taset, the std::ios::ate does not
> influence anything, IOW it's useless because writers repositions file
> pointer internally (calls seekp)
>
> ate - (at end) Set the stream's position indicator to the end of the
> stream on opening.
>
>
> Summarizing, there seem to be 3 sources of number of points possible:
> 1. call of LASHeader::SetPointRecordsCount
> 2. number of actually written files (WriterImpl::m_pointCount)
> 3. optionally, number of records specified in header block in a file
> being updated.
>
> Current version supports 1 and 2.
> If we want to support 3, we will have to make LASWriter able to read
> number of records from header and compare to 1 and 2.
> But:
> 1. How do we know which value is correct, or we assume it's 2 + 3
> 2. It completely breaks the idea of LASWriter which is supposed to  
> write
> data but not to edit existing data. (it's similar to GEOS, it writes  
> or
> reads but does not edit geometries).
>
> I'd suggest to move the update feature on the level of LASFile,  
> where we
> encapsulate access modes, reader and writer and potentially are able  
> to
> read something (header data) from a file that is opened in write mode.
>

Mateusz,

I have implemented a modified version that behaves as follows for  
WriterImpl and LASFile:

1) If the file is opened for write (eWrite in LASFile, which has mode  
std::ios::out | std::ios::binary), the existing overwriting behavior  
is preserved.
2) If the file is opened for append (eAppend in LASFile, which has  
mode std::ios::out | std::ios::in | std::ios::binary), we seek to the  
end and to the beginning of the file to determine how long it is,  
subtract the header, and then divide by the data record length to  
determine how many points (m_pointCount) the WriterImpl has.   Then,  
at the end of WriteHeader, if we have m_pointCount, seek to the end of  
the file so we can start writing, otherwise, leave it where it is.  
IMO, we can't necessarily believe the header's point count, and while  
the seeking may have a cost, this approach allows us to determine the  
number of points without actually reading all of them.
3) Added eAppend to LASFile and modified the FileImpl to take in a  
mode which it uses to determine the openmode to use.
4) Modified all of the calls to LASFile to set the mode in the  
constructor.

http://liblas.org/changeset/478

Howard