[GRASS-user] v.vol.rst - not enough disk space to write temp file

Mon Jan 14 17:19:42 PST 2019

it finally finished the process.  the files size that was created on the
external disk was 50GB as you can see on the attached screenshot.
the interpolation took 49 min to complete.

(Mon Jan 14 19:28:05 2019)

v.vol.rst input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2 dmin=2
elevation=dec16jan14
280957 records selected from table
WARNING: Some points outside of region -- will ignore...
WARNING: Strip exists with insufficient data
Processing all selected output files will require 836000000 bytes of disk
space for temp files
WARNING: There are points outside specified 2D/3D region--ignored 55094
points (total points: 280957)
WARNING: Points are more dense than specified 'DMIN'--ignored 156188 points
(remain 124769)
WARNING: Unable to create 'cross_output' raster map without 'cross_input'
raster map being specified
(Mon Jan 14 20:17:49 2019) Command finished (49 min 43 sec)

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Le lun. 14 janv. 2019 à 20:13, Francois Chartier <fra.chartier at gmail.com> a
écrit :

> grass data is on the external drive, see screenshot attached.
> freespace on external drive 1.77TB
> what is interesting with the external drive is the temp file does not
> really explode in size like if files are on laptop:
> it has been running for about 50 min now.
>
> I changed the region, made the domain slightly smaller and used a lower
> resolution:100x100x5
> g.region
> (Mon Jan 14 19:26:20 2019)
>
> g.region n=4875000 s=4820000 e=680000 w=585000 t=350 b=50 nsres=100
> ewres=100 tbres=5
> (Mon Jan 14 19:26:21 2019) Command finished (0 sec)
>
> for some reason it seems that i have 2 nsres and ewres
> C:\Users\Francois Chartier>g.region -p3
> projection: 1 (UTM)
> zone:       17
> datum:      nad83
> ellipsoid:  grs80
> north:      4875000
> south:      4820000
> west:       585000
> east:       680000
> top:        350.00000000
> bottom:     50.00000000
> nsres:      100
> nsres3:     5
> ewres:      100
> ewres3:     5
> tbres:      5
> rows:       550
> rows3:      11000
> cols:       950
> cols3:      19000
> depths:     60
> cells:      522500
> cells3:     12540000000
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> <#m_288032140617036686_m_-995905413045375258_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> Le dim. 13 janv. 2019 à 08:52, Markus Neteler <neteler at osgeo.org> a
> écrit :
>
>> Hi,
>>
>> On Sat, Jan 12, 2019 at 11:56 PM Francois Chartier
>> <fra.chartier at gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I checked the temp file that is saved during the v.vol.rst
>> interpolation process, and the file went up to over 50 GB and more until
>> the space on my laptop (free space of 100GB) was exausted.
>>
>> I start to wonder if those tmp files created are in any way compressed
>> (probably not?). This may be worth a bug ticket at
>> https://trac.osgeo.org/grass/newticket
>>
>> > So I copied all the grass data file onto an external hard drive with 1
>> Terabyte of space. started grass and browsed to the data that is
>> > saved on the external hard drive.  when running vvolrst, the process
>> also crashed due to not enough space....
>> > the memory situation is much better during the process.
>> > for some reason it does not seem to increase the size of the temp file
>> on the hard drive...
>>
>> So the grassdata directory with the location and mapset in question is
>> on the terabyte disk?
>>
>> > see error message below.
>> > (Sat Jan 12 17:28:02 2019)
>> > v.vol.rst --overwrite input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2
>> dmin=1 elevation=dec16jan12
>> > 280957 records selected from table
>> > WARNING: Some points outside of region -- will ignore...
>> > Processing all selected output files will require 2134724556 bytes of
>> disk space for temp files
>> > WARNING: There are points outside specified 2D/3D region--ignored 1
>> points (total points: 280957)
>> > WARNING: Points are more dense than specified 'DMIN'--ignored 158199
>> points (remain 122758)
>> > WARNING: Unable to create 'cross_output' raster map without
>> 'cross_input' raster map being specified
>> > ERROR: Not enough disk space - cannot write temp files
>>
>> I think we should try to improve the error message, telling on which
>> device this occurs.
>>
>> The code in question is:
>> vector/v.vol.rst/main.c, line 633
>>
>>             /* filling temp file with zeroes */
>>             for (i = 0; i < n_rows; i++) {
>>                 if (!
>>                     (fwrite
>>                      (zero_array_cell, sizeof(FCELL), n_cols,
>> Tmp_fd_cell))) {
>>                     clean();
>>                     G_fatal_error(_("Not enough disk space - cannot
>> write temp files"));
>>                 }
>>             }
>>
>> Does anyone know how to get a better error message?
>> Would this help?
>> https://linux.die.net/man/3/explain_fwrite
>>
>> > (Sat Jan 12 17:54:14 2019) Command finished (26 min 12 sec)
>> >
>> > below is the g.region -p3 printed, as you can see this is a large
>> domain and high resolution.
>> >
>> > projection: 1 (UTM)
>> > zone:       17
>> > datum:      nad83
>> > ellipsoid:  grs80
>> > north:      4924870
>> > south:      4784003
>> > west:       585000
>> > east:       679717
>> > top:        453.00000000
>> > bottom:     2.16080000
>> > nsres:      50.00603479
>> > nsres3:     5.00007099
>> > ewres:      50.00897571
>> > ewres3:     5.00010558
>> > tbres:      0.999643458980044
>>
>> Are the pixels/voxels non-square on purpose? This could be adjusted
>> with g.region.
>>
>> > rows:       2817
>> > rows3:      28173
>> > cols:       1894
>> > cols3:      18943
>> > depths:     451
>> > cells:      5335398
>> > cells3:     240690193689
>>
>> That is still a lot (just guessing around):
>> 240,690,193,689 raster3d_cells * 64 bit (but it will be larger than
>> 32bit) = 1.540417e+13 bit
>>
>> So, in the first place we need to know where GRASS GIS really writes
>> to in your system.
>>
>> Any devs having a suggestion here?
>>
>> Markus
>>
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> <#m_288032140617036686_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20190114/703b2c52/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tmp file.jpg
Type: image/jpeg
Size: 312203 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20190114/703b2c52/attachment-0001.jpg>