[GRASS-user] Fwd: skipping MEM driver with r.out.gdal for large files

[Markus Neteler]

Hi Laura,

On Thu, Jun 15, 2023 at 5:00 PM Laura Poggio <laura.poggio at gmail.com> wrote:
>
> Dear all,
> Apologies, sent to the developer mailing list by mistake.
> I am trying to save a rather large file from GRASS
>
> Versions:
> GRASS GIS 8.2.1
> GDAL 3.6.3, released 2023/03/07
> Linux: almalinux:9.1
>
> The size of the file is:
> rows=141120
> cols=362880
> cells=51209625600
>
> This the command
> r.out.gdal -c input=out output=output.tif format=COG type=Int16 createopt=COMPRESS=LZW,PREDICTOR=2,BIGTIFF=YES --overwrite --quiet
>
> This the output
> Warning 6: driver MEM does not support creation option COMPRESS
> Warning 6: driver MEM does not support creation option PREDICTOR
> Warning 6: driver MEM does not support creation option BIGTIFF
> ERROR 2: /usr/gdal-3.6.2/frmts/mem/memdataset.cpp, 1289: cannot allocate 1x102419251200 bytes
> ERROR: Unable to create dataset using memory raster driver
>
> Is there any option not to use the MEM driver when writing to disk?

I am not sure, in this code section it doesn't look like this:

https://github.com/OSGeo/grass/blob/51e962abbc1176211ad2c4cb7857a61a2b6d1c6f/raster/r.out.gdal/main.c#L377

Maybe someone else in the list has more insights?

Some options:
- try first the GTiff driver, then convert to COG with gdal_translate
- enlarge RAM using a swap file (no reboot needed but sudo access
needed) as disk space is relatively cheap and r.out.gdal will not
realize then that it gets swapped.

Best,
Markus

-- 
Markus Neteler, PhD
https://www.mundialis.de - free data with free software
https://grass.osgeo.org
https://courses.neteler.org/blog