[GRASS-user] v.vol.rst - not enough disk space to write temp file

[Markus Neteler]

Hi,

On Sat, Jan 12, 2019 at 11:56 PM Francois Chartier
<fra.chartier at gmail.com> wrote:
>
> Hi,
>
> I checked the temp file that is saved during the v.vol.rst interpolation process, and the file went up to over 50 GB and more until the space on my laptop (free space of 100GB) was exausted.

I start to wonder if those tmp files created are in any way compressed
(probably not?). This may be worth a bug ticket at
https://trac.osgeo.org/grass/newticket

> So I copied all the grass data file onto an external hard drive with 1 Terabyte of space. started grass and browsed to the data that is
> saved on the external hard drive.  when running vvolrst, the process also crashed due to not enough space....
> the memory situation is much better during the process.
> for some reason it does not seem to increase the size of the temp file on the hard drive...

So the grassdata directory with the location and mapset in question is
on the terabyte disk?

> see error message below.
> (Sat Jan 12 17:28:02 2019)
> v.vol.rst --overwrite input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2 dmin=1 elevation=dec16jan12
> 280957 records selected from table
> WARNING: Some points outside of region -- will ignore...
> Processing all selected output files will require 2134724556 bytes of disk space for temp files
> WARNING: There are points outside specified 2D/3D region--ignored 1 points (total points: 280957)
> WARNING: Points are more dense than specified 'DMIN'--ignored 158199 points (remain 122758)
> WARNING: Unable to create 'cross_output' raster map without 'cross_input' raster map being specified
> ERROR: Not enough disk space - cannot write temp files

I think we should try to improve the error message, telling on which
device this occurs.

The code in question is:
vector/v.vol.rst/main.c, line 633

            /* filling temp file with zeroes */
            for (i = 0; i < n_rows; i++) {
                if (!
                    (fwrite
                     (zero_array_cell, sizeof(FCELL), n_cols, Tmp_fd_cell))) {
                    clean();
                    G_fatal_error(_("Not enough disk space - cannot
write temp files"));
                }
            }

Does anyone know how to get a better error message?
Would this help?
https://linux.die.net/man/3/explain_fwrite

> (Sat Jan 12 17:54:14 2019) Command finished (26 min 12 sec)
>
> below is the g.region -p3 printed, as you can see this is a large domain and high resolution.
>
> projection: 1 (UTM)
> zone:       17
> datum:      nad83
> ellipsoid:  grs80
> north:      4924870
> south:      4784003
> west:       585000
> east:       679717
> top:        453.00000000
> bottom:     2.16080000
> nsres:      50.00603479
> nsres3:     5.00007099
> ewres:      50.00897571
> ewres3:     5.00010558
> tbres:      0.999643458980044

Are the pixels/voxels non-square on purpose? This could be adjusted
with g.region.

> rows:       2817
> rows3:      28173
> cols:       1894
> cols3:      18943
> depths:     451
> cells:      5335398
> cells3:     240690193689

That is still a lot (just guessing around):
240,690,193,689 raster3d_cells * 64 bit (but it will be larger than
32bit) = 1.540417e+13 bit

So, in the first place we need to know where GRASS GIS really writes
to in your system.

Any devs having a suggestion here?

Markus