[GRASS-dev] [GRASS GIS] #3433: i.zc: unable to handle large maps as everything is done in memory

GRASS GIS trac at osgeo.org
Thu Nov 2 07:45:56 PDT 2017


#3433: i.zc: unable to handle large maps as everything is done in memory
------------------------------+-------------------------
 Reporter:  mlennert          |      Owner:  grass-dev@…
     Type:  enhancement       |     Status:  new
 Priority:  normal            |  Milestone:  7.4.0
Component:  Imagery           |    Version:  svn-trunk
 Keywords:  i.zc large files  |        CPU:  Unspecified
 Platform:  Unspecified       |
------------------------------+-------------------------
 i.zc fails already in the data initialization stage when trying to analyze
 a large map (227,537,282 cells in the region):

 {{{
 i.zc pan out=zctmp width=9 thresh=1
 Power 2 values : 16384 rows 32768 columns
 Initializing data...
 Killed
 }}}

 IIUC, this comes from the fact that everything is loaded into memory,
 starting with just the initialization of vectors at the beginning which is
 where it fails:

 {{{
     /* get the rows and columns in the current window */
     or = Rast_window_rows();
     oc = Rast_window_cols();
     rows = G_math_max_pow2((long)or);
     cols = G_math_max_pow2((long)oc);
     size = (rows > cols) ? rows : cols;
     totsize = size * size;

     G_message(_("Power 2 values : %d rows %d columns"), rows, cols);

     /* Allocate appropriate memory for the structure containing
        the real and complex components of the FFT.  DATA[0] will
        contain the real, and DATA[1] the complex component.
      */
     data[0] = (double *)G_malloc(totsize * sizeof(double));
     data[1] = (double *)G_malloc(totsize * sizeof(double));

     /* Initialize real & complex components to zero */
     G_message(_("Initializing data..."));
     for (i = 0; i < (totsize); i++) {
         *(data[0] + i) = 0.0;
         *(data[1] + i) = 0.0;
     }
 }}}


 Further on, the module calls functions such as
 [https://trac.osgeo.org/grass/browser/grass/trunk/lib/gmath/del2g.c#L46
 del2g()] and
 [https://trac.osgeo.org/grass/browser/grass/trunk/lib/gmath/findzc.c#L56
 G_math_findzc] which use these data structures.

 I have no idea what it would entail to rewrite this so as to be able to
 deal with larger maps.

 At this stage I'm classifying this as an enhancement request as the module
 works as such, but I consider that GRASS GIS should not bail out on larger
 maps, so am close to considering this as a bug...

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3433>
GRASS GIS <https://grass.osgeo.org>



More information about the grass-dev mailing list