[GRASS5] i.fft segemntation fault

[Glynn Clements]

Stephan Holl wrote:

> > > But I figured out, that reducing the size of a window will make
> > > i.fft work.
> > > My original regions, where it segfaults:
> > 
> > > rows:       11923
> > > cols:       25151
> > 
> > > rows:       5445
> > > cols:       6519
> > 
> > 11923 * 25151 * 16 bytes (2 doubles) = 4798005968
> > 
> > I.e. ~4.8Gb, which is likely to significantly exceed the amount of
> > available memory (and, on 32-bit platforms, the amount of memory which
> > can be allocated to a single process, regardless of how much total RAM
> > or swap is available).
> > 
> > Actually, you would need half as much again (i.e. ~7.2Gb); i.fft reads
> > the entire map into an array of doubles (8 bytes per cell), but FFTW
> > needs an array of fftw_complex values (2 doubles, 16 bytes per cell).

Two other points which I overlooked:

1. i.fft assumes that the fft() function requires that the array
dimensions are powers of two (that was true for the old FFT routine,
but isn't true for FFTW), and allocates memory accordingly.

2. i.fft allocates space for both the real and imaginary components,
so it's actually four doubles (32 bytes) per cell.

So, you would actually need:

	16384 * 32768 * 32 bytes (4 doubles) = 17179869184 (16Gb)

for the original resolution, or:

	8192 * 8192  * 32 bytes (4 doubles) = 2147483648 (2Gb)

for the reduced resolution.

BTW, the reason why you need two copies is that the original fft()
function took separate real and imaginary arrays, while FFTW requires
an array of fftw_complex structs (i.e. real and imaginary components
are interleaved).

The memory requirements could be reduced to two doubles per cell
(without scaling up to a power of two) if i.fft used FFTW directly,
rather than using the fft() interface. Or to two floats per cell if
FFTW was built to use float instead of double.

> > More generally, maps that large won't work with anything which needs
> > to read the entire map into memory (which includes i.fft), unless you
> > have a 64-bit system and a lot of RAM (swap wouldn't be much use on
> > something like an FFT calculation).
> 
> so the only possible solution would be tiling the map into smaller
> pices which fit into my RAM (768MB) and calculating the fft on the small
> tiles.

Or, if you don't need the highest frequencies, just reduce the region
resolution.

-- 
Glynn Clements <glynn.clements at virgin.net>