[GRASS-dev] Error reading raster data for row xxx (only when using r.series and t.rast.series)

[Dylan Beaudette]

On Wed, Oct 14, 2015 at 10:50 AM, Dylan Beaudette
<dylan.beaudette at gmail.com> wrote:
> Some additional clues:
>
> The original stack was 365 maps with 3105 x 7025 cells.
>
> 1. zooming into a smaller region (30 x 40 cells) and running
> t.rast.series 100x resulted in 100 "correct" maps, no errors.
>
> 2. returning to the full extent and running t.rast.series 30x on the
> first 31 maps resulted in 30 "correct" maps, no errors.
>
> 3. returning to the full extent and running t.rast.series 30x on the
> last 31 maps resulted in 30 "correct" maps, no errors
>
>
> So, it seems that t.rast.series (r.series) is throwing an error, or
> generating wront output, when when:
>
> a large set of maps are supplied as input, and, a region that has a
> moderate number of total cells.
>
> Yeah, I know, that isn't very specific. I will try re-compiling with
> debugging and no optimization next.
>
> Dylan
>
>

More data,

1. re-compiled with CFLAGS="-g -Wall":
 * Multiple runs of t.rast.series with the full stack (365 maps with
3105 x 7025 cells), no errors.
 * each run required about 8.5 minutes to complete

2. re-compiled with  CFLAGS="-O2 -mtune=native -march=native" LDFLAGS="-s":
 * 10x tests with full stack, no errors
 * each run required about 3.5 minutes

3. re-run original script (see listing below)
 * random errors from t.rast.series

This doesn't make much sense to me. The only difference between my
latest "tests" and the original code is that the input to
t.rast.series was static over the course of my "tests", vs. dynamic
within the original code (see below). I purposely selected a stack
that caused t.rast.series to throw an error for my tests.

Arg!

Dylan


For the record, here is the script that I have been using:

years=`seq 1981 2010`
for year in $years
  do
  echo $year

  # make a where clause for finding the current year
  wc="strftime('%Y', start_time) = '"$year"'"

  # extract current dataset: "copies" by reference
  t.rast.extract --q --o input=tmin output=tmin_subset
basename=tmin_subset where="$wc"
  t.rast.extract --q --o input=tmax output=tmax_subset
basename=tmax_subset where="$wc"

  # compute GDD on each day (about 36 minutes)
  # values less than 0 are set to 0
  # NOTE: 4 CPUs so that external disk isn't thrashed
  gdd_max_C=30
  gdd_min_C=10
  gdd_base_C=10
  t.rast.mapcalc --q --o nprocs=4 input=tmin_subset,tmax_subset
output=gdd basename=gdd expr="max(((min(tmax_subset, $gdd_max_C) +
max(tmin_subset, $gdd_min_C)) / 2.0) - $gdd_base_C, 0)"

  # save Sonora time-series for later
  # -k flag ensures that the output is in same order as input
  t.rast.list -s gdd columns="id" | parallel -k --gnu r.what map="{}"
points=sonora > qa_qc/sonora-daily-gdd-$year.dat

  # sum GDD for this year: fast from SSD, but only single CPU thread
(3.5 minutes)
  # BUG: crashes randomly
  t.rast.series --q --o in=gdd out=gdd_$year method=sum

  done