[gdal-dev] Filelimit /vsizip

[Even Rouault]

Le jeudi 13 novembre 2014 17:47:26, Julian Zeidler a écrit :
> Hi all,
> I am wondering if there is an inherent Filelimit in the /vsizip virtual
> Interface (or potentially ENVI Format)
> I am trying to concurrently open a lot (>600) ENVI Files, which are
> inside single image zip Files via the /vsizip interface.
> Whenever i hit the Limit of exactly 510 images I get an:

Julian,

2 * 510 = 1020 < 1024, the default maximum number of files that can be opened 
simultaneously by a Linux process. The 2 factor comes from the fact that a 
ENVI dataset is made of an .img and .hdr file.
So nothing GDAL can do about that.
If you had admin rights, you could raise the limit with ulimit -n
Otherwise you will just have to avoid opening so many files at the same time.

An optimization in the ENVI driver could likely be done to close the .hdr file 
once we have read it. But that would raise the limit "just" up to 1000 
datasets.

Well, one option would be to use the GDALProxyPoolDataset mechanism, but this 
is a rather advanced/internal/undocumented feature used by the VRT driver to 
be able to deal with an arbitrary number of sources. If you badly need it, you 
may study gcore/gdal_proxy.h and look how it used by frmts/vrt/vrtsources.cpp.

Even

> "ERROR 4: Failed to re-open
> /vsizip//mnt/data1/download_asar/ASA_WSM_1PNPDE20110731_205400_000002143105
> _00100_49252_3499.zip/ASA_WSM_1PNPDE20110731_205400_000002143105_00100_4925
> 2_3499.data/Sigma0_VV.img within ENVI driver."
> I am running gdal 1.9.0 on a Debian Linux from within a c++ Programm
> 
> Is there any way to increase this Limit? Preferably without having to
> recompile gdal, as i do not have Admin rights on the servers it runs on?
> 
> Thanks
> Julian
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com