[GRASS-dev] [GRASS GIS] #2428: r.external to accept gdal config options

[GRASS GIS]

#2428: r.external to accept gdal config options
--------------------------+-------------------------
  Reporter:  perrygeo     |      Owner:  grass-dev@…
      Type:  enhancement  |     Status:  new
  Priority:  normal       |  Milestone:  7.1.0
 Component:  Default      |    Version:  svn-trunk
Resolution:               |   Keywords:  r.external
       CPU:  x86-64       |   Platform:  Unspecified
--------------------------+-------------------------

Comment (by dylan):

 Replying to [comment:3 neteler]:
 > Replying to [comment:2 glynn]:
 > > Replying to [comment:1 neteler]:
 > > > The request sounds reasonable but I didn't figure out how to pass
 "papszOptions" to GDAL as used in r.external. The magic line in r.out.gdal
 is this:
 > >
 > > > but in r.external GDALOpen() is used. Perhaps a GDAL expert can tell
 us the trick.
 > >
 > > Note that r.external is the analogue of r.in.gdal, which doesn't
 accept any configuration options.
 >
 > While it doesn't directly, I had added a larger cache some time ago by
 setting GDALSetCacheMax() to 300MB rather than the tiny 40MB default GDAL
 cache size. This speeds up import tremendously:
 >
 > {{{
 > r.in.gdal ...
 > memory=integer
 >     Maximum memory to be used (in MB)
 >     Cache size for raster rows
 >     Options: 0-2047
 >     Default: 300
 > }}}
 >
 > I wonder how to get that into r.external (I suppose that it would
 benefit as well).
 >
 > > If options are needed for reading, a similar option should be added to
 both r.in.gdal and r.external, presumably using GDALOpenEx() instead of
 GDALOpen(). The latter will require extending the GDAL "link" format
 (lib/raster/gdal.c).
 >
 > OK (no idea how to implement that).

 Finding this thread after searching for some ways to speed-up file access
 to maps linked via `r.external`.

 The adjustable cache solution in `r.in.gdal` appears to be:

 {{{
 if (parm.memory->answer && *parm.memory->answer) {
            /* TODO: GDALGetCacheMax() overflows at 2GiB, implement use of
 GDALSetCacheMax64() */
            GDALSetCacheMax(atol(parm.memory->answer) * 1024 * 1024);
            G_verbose_message(_("Using memory cache size: %.1f MiB"),
 GDALGetCacheMax()/1024.0/1024.0);
     }
 }}}


 Could this same block of code be used within `r.external`? I don't fully
 understand how `r.external` works, so I suppose that it is more
 complicated than this.

 Or another option, is there an environmental variable that could be used
 to control the GDAL cache size?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2428#comment:4>
GRASS GIS <https://grass.osgeo.org>