[GRASS-user] v.vol.rst - not enough disk space to write temp file

Tue Jan 15 05:31:15 PST 2019

i will try with a smaller model domain to see if the size of the domain the
source of the problem.

On Mon, Jan 14, 2019, 20:19 Francois Chartier <fra.chartier at gmail.com wrote:

> it finally finished the process.  the files size that was created on the
> external disk was 50GB as you can see on the attached screenshot.
> the interpolation took 49 min to complete.
>
>
> (Mon Jan 14 19:28:05 2019)
>
> v.vol.rst input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2 dmin=2
> elevation=dec16jan14
> 280957 records selected from table
> WARNING: Some points outside of region -- will ignore...
> WARNING: Strip exists with insufficient data
> Processing all selected output files will require 836000000 bytes of disk
> space for temp files
> WARNING: There are points outside specified 2D/3D region--ignored 55094
> points (total points: 280957)
> WARNING: Points are more dense than specified 'DMIN'--ignored 156188
> points (remain 124769)
> WARNING: Unable to create 'cross_output' raster map without 'cross_input'
> raster map being specified
> (Mon Jan 14 20:17:49 2019) Command finished (49 min 43 sec)
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> <#m_-6629092632311730313_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> Le lun. 14 janv. 2019 à 20:13, Francois Chartier <fra.chartier at gmail.com>
> a écrit :
>
>> grass data is on the external drive, see screenshot attached.
>> freespace on external drive 1.77TB
>> what is interesting with the external drive is the temp file does not
>> really explode in size like if files are on laptop:
>> it has been running for about 50 min now.
>>
>> I changed the region, made the domain slightly smaller and used a lower
>> resolution:100x100x5
>> g.region
>> (Mon Jan 14 19:26:20 2019)
>>
>> g.region n=4875000 s=4820000 e=680000 w=585000 t=350 b=50 nsres=100
>> ewres=100 tbres=5
>> (Mon Jan 14 19:26:21 2019) Command finished (0 sec)
>>
>> for some reason it seems that i have 2 nsres and ewres
>> C:\Users\Francois Chartier>g.region -p3
>> projection: 1 (UTM)
>> zone:       17
>> datum:      nad83
>> ellipsoid:  grs80
>> north:      4875000
>> south:      4820000
>> west:       585000
>> east:       680000
>> top:        350.00000000
>> bottom:     50.00000000
>> nsres:      100
>> nsres3:     5
>> ewres:      100
>> ewres3:     5
>> tbres:      5
>> rows:       550
>> rows3:      11000
>> cols:       950
>> cols3:      19000
>> depths:     60
>> cells:      522500
>> cells3:     12540000000
>>
>>
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
>> www.avast.com
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>> <#m_-6629092632311730313_m_288032140617036686_m_-995905413045375258_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>
>> Le dim. 13 janv. 2019 à 08:52, Markus Neteler <neteler at osgeo.org> a
>> écrit :
>>
>>> Hi,
>>>
>>> On Sat, Jan 12, 2019 at 11:56 PM Francois Chartier
>>> <fra.chartier at gmail.com> wrote:
>>> >
>>> > Hi,
>>> >
>>> > I checked the temp file that is saved during the v.vol.rst
>>> interpolation process, and the file went up to over 50 GB and more until
>>> the space on my laptop (free space of 100GB) was exausted.
>>>
>>> I start to wonder if those tmp files created are in any way compressed
>>> (probably not?). This may be worth a bug ticket at
>>> https://trac.osgeo.org/grass/newticket
>>>
>>> > So I copied all the grass data file onto an external hard drive with 1
>>> Terabyte of space. started grass and browsed to the data that is
>>> > saved on the external hard drive.  when running vvolrst, the process
>>> also crashed due to not enough space....
>>> > the memory situation is much better during the process.
>>> > for some reason it does not seem to increase the size of the temp file
>>> on the hard drive...
>>>
>>> So the grassdata directory with the location and mapset in question is
>>> on the terabyte disk?
>>>
>>> > see error message below.
>>> > (Sat Jan 12 17:28:02 2019)
>>> > v.vol.rst --overwrite input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2
>>> dmin=1 elevation=dec16jan12
>>> > 280957 records selected from table
>>> > WARNING: Some points outside of region -- will ignore...
>>> > Processing all selected output files will require 2134724556 bytes of
>>> disk space for temp files
>>> > WARNING: There are points outside specified 2D/3D region--ignored 1
>>> points (total points: 280957)
>>> > WARNING: Points are more dense than specified 'DMIN'--ignored 158199
>>> points (remain 122758)
>>> > WARNING: Unable to create 'cross_output' raster map without
>>> 'cross_input' raster map being specified
>>> > ERROR: Not enough disk space - cannot write temp files
>>>
>>> I think we should try to improve the error message, telling on which
>>> device this occurs.
>>>
>>> The code in question is:
>>> vector/v.vol.rst/main.c, line 633
>>>
>>>             /* filling temp file with zeroes */
>>>             for (i = 0; i < n_rows; i++) {
>>>                 if (!
>>>                     (fwrite
>>>                      (zero_array_cell, sizeof(FCELL), n_cols,
>>> Tmp_fd_cell))) {
>>>                     clean();
>>>                     G_fatal_error(_("Not enough disk space - cannot
>>> write temp files"));
>>>                 }
>>>             }
>>>
>>> Does anyone know how to get a better error message?
>>> Would this help?
>>> https://linux.die.net/man/3/explain_fwrite
>>>
>>> > (Sat Jan 12 17:54:14 2019) Command finished (26 min 12 sec)
>>> >
>>> > below is the g.region -p3 printed, as you can see this is a large
>>> domain and high resolution.
>>> >
>>> > projection: 1 (UTM)
>>> > zone:       17
>>> > datum:      nad83
>>> > ellipsoid:  grs80
>>> > north:      4924870
>>> > south:      4784003
>>> > west:       585000
>>> > east:       679717
>>> > top:        453.00000000
>>> > bottom:     2.16080000
>>> > nsres:      50.00603479
>>> > nsres3:     5.00007099
>>> > ewres:      50.00897571
>>> > ewres3:     5.00010558
>>> > tbres:      0.999643458980044
>>>
>>> Are the pixels/voxels non-square on purpose? This could be adjusted
>>> with g.region.
>>>
>>> > rows:       2817
>>> > rows3:      28173
>>> > cols:       1894
>>> > cols3:      18943
>>> > depths:     451
>>> > cells:      5335398
>>> > cells3:     240690193689
>>>
>>> That is still a lot (just guessing around):
>>> 240,690,193,689 raster3d_cells * 64 bit (but it will be larger than
>>> 32bit) = 1.540417e+13 bit
>>>
>>> So, in the first place we need to know where GRASS GIS really writes
>>> to in your system.
>>>
>>> Any devs having a suggestion here?
>>>
>>> Markus
>>>
>>
>>
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
>> www.avast.com
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>> <#m_-6629092632311730313_m_288032140617036686_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20190115/f1cc5dc5/attachment.html>