[GRASS5] 5.4 release branch

Glynn Clements glynn.clements at virgin.net
Wed Oct 20 12:44:04 EDT 2004


Paul Kelly wrote:

> >>> Raster I/O re-write incorporating support for files > 2GB (Glynn)
> >>
> >> Probably it should become a flag for 'configure' (maybe that is already?)?
> >
> > I'm not sure about the right way to implement this. Ideally, it should
> > only be the core raster I/O files which I changed which should be
> > using the large-file support, as the rest of GRASS won't handle large
> > files correctly.
> 
> I think maybe we should put support for it into the 5.7 configure script, 
> encourage people to use 5.7 if they want large raster file support, and 
> leave it as an undocumented unsupported feature of 5.4 for now.

The raster I/O files ({get,put}_row.c etc) are shared between 5.3 and
5.7.

The complication isn't in adding a configure option, it's deciding how
to use it.

If you want to enable large file support globally, you can already do
this with e.g.:

	CFLAGS=-D_FILE_OFFSET_BITS=64 ./configure ...

However, most of GRASS can't handle large files, as file offsets are
stored and calculated using 32 bits; offsets beyond 2Gb will be
silently "wrapped".

Unless someone is willing to track down all of the code which uses
lseek(), and fix it to use off_t (including in any intermediate
results), globally enabling large file support will be dangerous.

As that (fixing lseek() usage) is unlikely to happen soon, we would
need to decide on a mechanism for using it selectively. Also, only
supporting large rasters internally has its own problems, e.g. you may
not be able to export them with certain programs, or use them from
modules which copy the data to a temporary file (although that's
already an issue; due to compression, a raster which occupies less
than 2Gb internally may require more than 2Gb when uncompressed).

-- 
Glynn Clements <glynn.clements at virgin.net>




More information about the grass-dev mailing list