[GRASS-user] v.vol.rst - not enough disk space to write temp file
Francois Chartier
fra.chartier at gmail.com
Wed Jan 16 19:00:04 PST 2019
My solution is to use a smaller model domain with 'only' 16 million cells,
with grid resolution of 10 x 10 m horizontally, and 1 m vertically.
is there anyway to remove the shade in the 3D view ?
Le mar. 15 janv. 2019 à 08:31, Francois Chartier <fra.chartier at gmail.com> a
écrit :
> i will try with a smaller model domain to see if the size of the domain
> the source of the problem.
>
> On Mon, Jan 14, 2019, 20:19 Francois Chartier <fra.chartier at gmail.com
> wrote:
>
>> it finally finished the process. the files size that was created on the
>> external disk was 50GB as you can see on the attached screenshot.
>> the interpolation took 49 min to complete.
>>
>>
>> (Mon Jan 14 19:28:05 2019)
>>
>> v.vol.rst input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2 dmin=2
>> elevation=dec16jan14
>> 280957 records selected from table
>> WARNING: Some points outside of region -- will ignore...
>> WARNING: Strip exists with insufficient data
>> Processing all selected output files will require 836000000 bytes of disk
>> space for temp files
>> WARNING: There are points outside specified 2D/3D region--ignored 55094
>> points (total points: 280957)
>> WARNING: Points are more dense than specified 'DMIN'--ignored 156188
>> points (remain 124769)
>> WARNING: Unable to create 'cross_output' raster map without 'cross_input'
>> raster map being specified
>> (Mon Jan 14 20:17:49 2019) Command finished (49 min 43 sec)
>>
>>
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
>> www.avast.com
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>> <#m_7934602069316905507_m_-6629092632311730313_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>
>> Le lun. 14 janv. 2019 à 20:13, Francois Chartier <fra.chartier at gmail.com>
>> a écrit :
>>
>>> grass data is on the external drive, see screenshot attached.
>>> freespace on external drive 1.77TB
>>> what is interesting with the external drive is the temp file does not
>>> really explode in size like if files are on laptop:
>>> it has been running for about 50 min now.
>>>
>>> I changed the region, made the domain slightly smaller and used a lower
>>> resolution:100x100x5
>>> g.region
>>> (Mon Jan 14 19:26:20 2019)
>>>
>>> g.region n=4875000 s=4820000 e=680000 w=585000 t=350 b=50 nsres=100
>>> ewres=100 tbres=5
>>> (Mon Jan 14 19:26:21 2019) Command finished (0 sec)
>>>
>>> for some reason it seems that i have 2 nsres and ewres
>>> C:\Users\Francois Chartier>g.region -p3
>>> projection: 1 (UTM)
>>> zone: 17
>>> datum: nad83
>>> ellipsoid: grs80
>>> north: 4875000
>>> south: 4820000
>>> west: 585000
>>> east: 680000
>>> top: 350.00000000
>>> bottom: 50.00000000
>>> nsres: 100
>>> nsres3: 5
>>> ewres: 100
>>> ewres3: 5
>>> tbres: 5
>>> rows: 550
>>> rows3: 11000
>>> cols: 950
>>> cols3: 19000
>>> depths: 60
>>> cells: 522500
>>> cells3: 12540000000
>>>
>>>
>>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
>>> www.avast.com
>>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>>> <#m_7934602069316905507_m_-6629092632311730313_m_288032140617036686_m_-995905413045375258_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>>
>>> Le dim. 13 janv. 2019 à 08:52, Markus Neteler <neteler at osgeo.org> a
>>> écrit :
>>>
>>>> Hi,
>>>>
>>>> On Sat, Jan 12, 2019 at 11:56 PM Francois Chartier
>>>> <fra.chartier at gmail.com> wrote:
>>>> >
>>>> > Hi,
>>>> >
>>>> > I checked the temp file that is saved during the v.vol.rst
>>>> interpolation process, and the file went up to over 50 GB and more until
>>>> the space on my laptop (free space of 100GB) was exausted.
>>>>
>>>> I start to wonder if those tmp files created are in any way compressed
>>>> (probably not?). This may be worth a bug ticket at
>>>> https://trac.osgeo.org/grass/newticket
>>>>
>>>> > So I copied all the grass data file onto an external hard drive with
>>>> 1 Terabyte of space. started grass and browsed to the data that is
>>>> > saved on the external hard drive. when running vvolrst, the process
>>>> also crashed due to not enough space....
>>>> > the memory situation is much better during the process.
>>>> > for some reason it does not seem to increase the size of the temp
>>>> file on the hard drive...
>>>>
>>>> So the grassdata directory with the location and mapset in question is
>>>> on the terabyte disk?
>>>>
>>>> > see error message below.
>>>> > (Sat Jan 12 17:28:02 2019)
>>>> > v.vol.rst --overwrite input=Dec16CleanedJjan12 at Toronto wcolumn=dbl_2
>>>> dmin=1 elevation=dec16jan12
>>>> > 280957 records selected from table
>>>> > WARNING: Some points outside of region -- will ignore...
>>>> > Processing all selected output files will require 2134724556 bytes of
>>>> disk space for temp files
>>>> > WARNING: There are points outside specified 2D/3D region--ignored 1
>>>> points (total points: 280957)
>>>> > WARNING: Points are more dense than specified 'DMIN'--ignored 158199
>>>> points (remain 122758)
>>>> > WARNING: Unable to create 'cross_output' raster map without
>>>> 'cross_input' raster map being specified
>>>> > ERROR: Not enough disk space - cannot write temp files
>>>>
>>>> I think we should try to improve the error message, telling on which
>>>> device this occurs.
>>>>
>>>> The code in question is:
>>>> vector/v.vol.rst/main.c, line 633
>>>>
>>>> /* filling temp file with zeroes */
>>>> for (i = 0; i < n_rows; i++) {
>>>> if (!
>>>> (fwrite
>>>> (zero_array_cell, sizeof(FCELL), n_cols,
>>>> Tmp_fd_cell))) {
>>>> clean();
>>>> G_fatal_error(_("Not enough disk space - cannot
>>>> write temp files"));
>>>> }
>>>> }
>>>>
>>>> Does anyone know how to get a better error message?
>>>> Would this help?
>>>> https://linux.die.net/man/3/explain_fwrite
>>>>
>>>> > (Sat Jan 12 17:54:14 2019) Command finished (26 min 12 sec)
>>>> >
>>>> > below is the g.region -p3 printed, as you can see this is a large
>>>> domain and high resolution.
>>>> >
>>>> > projection: 1 (UTM)
>>>> > zone: 17
>>>> > datum: nad83
>>>> > ellipsoid: grs80
>>>> > north: 4924870
>>>> > south: 4784003
>>>> > west: 585000
>>>> > east: 679717
>>>> > top: 453.00000000
>>>> > bottom: 2.16080000
>>>> > nsres: 50.00603479
>>>> > nsres3: 5.00007099
>>>> > ewres: 50.00897571
>>>> > ewres3: 5.00010558
>>>> > tbres: 0.999643458980044
>>>>
>>>> Are the pixels/voxels non-square on purpose? This could be adjusted
>>>> with g.region.
>>>>
>>>> > rows: 2817
>>>> > rows3: 28173
>>>> > cols: 1894
>>>> > cols3: 18943
>>>> > depths: 451
>>>> > cells: 5335398
>>>> > cells3: 240690193689
>>>>
>>>> That is still a lot (just guessing around):
>>>> 240,690,193,689 raster3d_cells * 64 bit (but it will be larger than
>>>> 32bit) = 1.540417e+13 bit
>>>>
>>>> So, in the first place we need to know where GRASS GIS really writes
>>>> to in your system.
>>>>
>>>> Any devs having a suggestion here?
>>>>
>>>> Markus
>>>>
>>>
>>>
>>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.
>>> www.avast.com
>>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>>> <#m_7934602069316905507_m_-6629092632311730313_m_288032140617036686_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20190116/5526d560/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smallerdomain.jpg
Type: image/jpeg
Size: 374141 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20190116/5526d560/attachment-0001.jpg>
More information about the grass-user
mailing list