[GRASS-user] r.in.xyz: Could open text file ~ 2.5GB

Glynn Clements glynn at gclements.plus.com
Fri Oct 20 10:12:58 EDT 2006


Michael Perdue wrote:

> Personally, I haven't had any LFS support problems with r.in.xyz.  
> This included scanning a text file for extents as your trying to do.  
> After the initial thread was started with respect to r.in.xyz's LFS I  
> went back and checked some of the test data sets that I have. They're  
> all beyond the 2GB limit (3.7GB is the largest i've tested so far)  
> and I haven't had any problems with the program. As far as I can see,  
> the only thing I have noticed that is different with a file beyond  
> the 2 gig size is that the progress indicator never comes up.

It sounds as if the OS maps fopen() to fopen64() (or uses O_LARGEFILE
in fopen()) regardless of the _FILE_OFFSET_BITS setting. In that case,
the fopen() will succeed regardless of the file's size, but the call
to fseek() will fail with EOVERFLOW.

Out of curiosity, which OS is that?

FWIW, this behaviour is a bad idea. If you can't reliably use
fseek/ftell on a file, fopen is supposed to fail when you try to open
the file, rather than succeeding then saying "ha! fooled you!" when
you try to read the current offset. If you are modifying an existing
file, failing half-way through (potentially leaving the file in an
inconsistent state) is a bad thing.

A correct call to fseek() shouldn't be *able* to fail on a normal file
opened with fopen(). You shouldn't have to worry about offset
overflows unless you use fopen64(), either literally or by using
-D_FILE_OFFSET_BITS=64 to remap the stdio functions to their 64-bit
equivalents.

-- 
Glynn Clements <glynn at gclements.plus.com>




More information about the grass-user mailing list