[gdal-dev] GDAL + Docker with NFS

[Even Rouault]

On dimanche 18 mars 2018 08:32:49 CET Andrew C Aitchison wrote:
> On Sun, 18 Mar 2018, Ouwen Huang wrote:
> > I feel quite silly, but it seems that the `gdal.Open` was actually opening
> > the file from the docker mounted NFS, just very slowly. I left it to run
> > for 2 minutes or so and it returned.
> > 
> > But this is weird behavior because on the host a `*Dataset` was returned
> > instantly, and within the docker container running `cp
> > /mnt/efs/<my_geo.tiff>  <docker_local_dir` is also instant. The 2min
> > latency seems to only occur within a docker mounted network filesystem for
> > the `gdal.Open` command.
> > 
> > Some other findings, the `/mnt/efs/` directory i'm using has several files
> > (100k+). When I move the geotiff under a new directory within EFS
> > (`mnt/efs/test`), the `gdal.Open` command is instant within the docker
> > container.
> > 
> > Is there a reason why `gdal.Open` is slow for a cluttered directory but
> > unix `cp` is not?
> 
> http://www.gdal.org/classGDALOpenInfo.html
> talks about "sibling" files; that is other files in the same directory.
> These are opened at the same time for some reason.
> 
> https://trac.osgeo.org/gdal/ticket/2158 suggests that the environment
> variable GDAL_DISABLE_READDIR_ON_OPEN should disable this.

I was going to suggest that as well. See
https://trac.osgeo.org/gdal/wiki/ConfigOptions#GDAL_DISABLE_READDIR_ON_OPEN

Try GDAL_DISABLE_READDIR_ON_OPEN=TRUE as environment variable.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com