[GRASS-user] Fwd: skipping MEM driver with r.out.gdal for large files

[Laura Poggio]

Dear Markus,
Using GTiff (in grass version 8.3.0 and with gdal 3.7.1) solved the
problem. I get a compressed file and the use of RAM is acceptable. I still
get the same errors when using COG.
A quick test with a different file in grass 8.2.1 and gdal 3.6.4 (previous
configuration) indicates that it would be the same there, but I could not
test it with the original file because of some hardware issues.

Thanks a lot!

Laura


On Thu, 22 Jun 2023 at 09:16, Markus Neteler <neteler at osgeo.org> wrote:

> Hi Laura,
>
> On Thu, Jun 15, 2023 at 5:00 PM Laura Poggio <laura.poggio at gmail.com>
> wrote:
> >
> > Dear all,
> > Apologies, sent to the developer mailing list by mistake.
> > I am trying to save a rather large file from GRASS
> >
> > Versions:
> > GRASS GIS 8.2.1
> > GDAL 3.6.3, released 2023/03/07
> > Linux: almalinux:9.1
> >
> > The size of the file is:
> > rows=141120
> > cols=362880
> > cells=51209625600
> >
> > This the command
> > r.out.gdal -c input=out output=output.tif format=COG type=Int16
> createopt=COMPRESS=LZW,PREDICTOR=2,BIGTIFF=YES --overwrite --quiet
> >
> > This the output
> > Warning 6: driver MEM does not support creation option COMPRESS
> > Warning 6: driver MEM does not support creation option PREDICTOR
> > Warning 6: driver MEM does not support creation option BIGTIFF
> > ERROR 2: /usr/gdal-3.6.2/frmts/mem/memdataset.cpp, 1289: cannot allocate
> 1x102419251200 bytes
> > ERROR: Unable to create dataset using memory raster driver
> >
> > Is there any option not to use the MEM driver when writing to disk?
>
> I am not sure, in this code section it doesn't look like this:
>
>
> https://github.com/OSGeo/grass/blob/51e962abbc1176211ad2c4cb7857a61a2b6d1c6f/raster/r.out.gdal/main.c#L377
>
> Maybe someone else in the list has more insights?
>
> Some options:
> - try first the GTiff driver, then convert to COG with gdal_translate
> - enlarge RAM using a swap file (no reboot needed but sudo access
> needed) as disk space is relatively cheap and r.out.gdal will not
> realize then that it gets swapped.
>
> Best,
> Markus
>
> --
> Markus Neteler, PhD
> https://www.mundialis.de - free data with free software
> https://grass.osgeo.org
> https://courses.neteler.org/blog
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20230727/6c2be9e5/attachment.htm>