[GRASS-dev] [GRASS GIS] #2350: G7: r.texture large file support problem

[Glynn Clements]

Vaclav Petras wrote:

> >  > Is r.texture supporting LFS?
> >
> >  Modules typically don't need to do anything regarding LFS; the support is
> >  in the libraries. The main issue which affects modules is that they
> >  shouldn't assume that cell counts will fit into an "int" or even a "long".
> >  But even failing to do so won't have any effect upon I/O.
> 
> Can you please explain what the modules are allowed to do? r.example does
> not tell [1].

The most common way for the issue to arise is multiplying the number
of rows by the number of columns to obtain the total number of cells. 
Most modules have no need to do this, but it is occasionally done e.g. 
when calculating statistics or storing the data in a temporary file.

The number of rows and columns can reasonably be assumed to fit into a
signed 32-bit integer, but their product cannot.

Even on a system with a 64-bit "long"[1], multiplying 2 "int"s will
produce an "int" result, and assigning the result to a "long" variable
doesn't change that. E.g.

	int nrows = Rast_window_rows();
	int ncols = Rast_window_cols();
	long ncells = nrows * ncols;

will truncate the result of the multiplication to an "int" (which is
32 bits on all mainstream platforms) then expand the truncated value
to 64 bits in the assignment. To perform the multplication using
"long", one of the arguments must be converted, e.g.:

	long ncells = (long) nrows * ncols;

[1] This doesn't include 64-bit versions of Windows, where "long" is
only 32 bits for compatibility reasons.

The issue isn't strictly related to LFS; due to compression, it's
possible for a raster with more than 2^31 cells to take up less than
2 GiB on disk. LFS just makes the issue more likely to arise in practice.

-- 
Glynn Clements <glynn at gclements.plus.com>