[GRASS-dev] [GRASS GIS] #2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce

[GRASS GIS]

#2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce
---------------------+-------------------------
  Reporter:  dylan   |      Owner:  grass-dev@…
      Type:  defect  |     Status:  new
  Priority:  normal  |  Milestone:  7.2.3
 Component:  Raster  |    Version:  unspecified
Resolution:          |   Keywords:
       CPU:  x86-64  |   Platform:  Linux
---------------------+-------------------------

Comment (by mmetz):

 Replying to [comment:34 mmetz]:
 > Replying to [comment:33 dylan]:
 > > I see that the SSD (sdd1) is idle for most of the time and then spikes
 to 30-80% of its "disk utilization" as reported by `dstat --disk-util`
 when several `r.sun` jobs finish (?).
 >
 > After `r.sun` comes `r.mapcalc` with a simple expresssion, producing
 output quite fast. This could be the reason for the spikes.

 The reason for the spikes in disk utilization is most likely `r.sun`
 because it writes out all output at once at the end.

 Coincidentally, I encountered similar problems on a different system where
 many GRASS commands were running in parallel. The errors were reading and
 decompression errors of both the data file and the null file. There were 2
 hosts doing the processing, the first host running 4 jobs in parallel, the
 second host running 8 jobs in parallel. Reading and decompression errors
 happened only on the first host running only 4 jobs in parallel. The
 problem was that the file system used for temporary data was nearly full
 and that it was not optimized for synchronous IO requests, slowing down
 the whole system and causing these errors. I freed some space on that
 filesystem and optimized its configuration, now all is running ok. Not
 perfect, the system is still sluggish in response, but I don't get any
 errors again.

 Therefore this ticket must IMHO be closed as wontfix because the solution
 is to adjust hardware, filesystems, and OS to heavy IO read/write
 requests.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2764#comment:35>
GRASS GIS <https://grass.osgeo.org>