[gdal-dev] Limit on the number of files for gdalbuildvrt

[Jose Gomez-Dans]

Dear Even,

Thanks for your help. After a bit more dredging, I am only able to repeat
my problem on a particularly mounted filesystem in an HPC facility I use.
On other systems (and even other mountpoints on that same HPC system)
things work as expected. I've taken this up with our administrators, but
just as a warning to others that it can happen and I didn't dream it!

Thanks for your very prompt reply
Jose

On Wed, 30 Oct 2019 at 16:16, Even Rouault <even.rouault at spatialys.com>
wrote:

> On mercredi 30 octobre 2019 15:52:54 CET Jose Gomez-Dans wrote:
> > Hi,
> >
> > I'm mosaicking a large number of small tiles using the GDAL python
> bindings
> > (python 3.7, gdal 3.0.1, but also tested on 2.4.2). Code is something
> like
> >
> > ```
> >
> > *output_fname = "/something/or/other.vrt"*
> > *fnames = ["file1.tif", ..., "fileN.tif"]*
> >
> >
> >
> >
> > *dst_ds = gdal.BuildVRT(            output_fname,            fnames,
> >     options=gdal.BuildVRTOptions(separate=True),        )*
> > *dst_ds = None*
> > ```
> >
> > This fails with lots of errors like
> > ERROR 4: fileXXX.tif: No such file or directory
> > Warning 1: Can't open fileXXX.tif. Skipping it
>
> gdalbuildvrt is normally cautious about non opening too many files at the
> same
> time. I can't replicate your issue with the following script
>
> ```
> from osgeo import gdal
> import shutil
>
> fnames = []
> for i in range(1024):
>     dstname = 'testbuildvrt/byte%d.tif' % i
>     shutil.copy('byte.tif', dstname)
>     fnames.append(dstname)
>
> dst_ds = gdal.BuildVRT(
>             "/tmp/out.vrt",
>             fnames,
>             options=gdal.BuildVRTOptions(separate=True),
>         )
> dst_ds = None
> ```
>
> > The aim of this is to create a VRT mosaic that can then be converted to
> > GeoTIFF.
>
> If you want a mosaic, you likely want separate=False.
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20191101/15bd20f4/attachment.html>