[gdal-dev] Filelimit /vsizip

[Julian Zeidler]

Hi Even,

Thanks to the reply.
I did not consider it an underlying Linux Problem, as I was opening 
vrts. with 6000+ Files without a Problem.
Thanks for the Info. I will look into having the Limit increased or in 
the Long run using the GDALProxyPoolDataset mechanism .
Julian

Am 13.11.2014 18:30, schrieb Even Rouault:
> Le jeudi 13 novembre 2014 17:47:26, Julian Zeidler a écrit :
>> Hi all,
>> I am wondering if there is an inherent Filelimit in the /vsizip virtual
>> Interface (or potentially ENVI Format)
>> I am trying to concurrently open a lot (>600) ENVI Files, which are
>> inside single image zip Files via the /vsizip interface.
>> Whenever i hit the Limit of exactly 510 images I get an:
> Julian,
>
> 2 * 510 = 1020 < 1024, the default maximum number of files that can be opened
> simultaneously by a Linux process. The 2 factor comes from the fact that a
> ENVI dataset is made of an .img and .hdr file.
> So nothing GDAL can do about that.
> If you had admin rights, you could raise the limit with ulimit -n
> Otherwise you will just have to avoid opening so many files at the same time.
>
> An optimization in the ENVI driver could likely be done to close the .hdr file
> once we have read it. But that would raise the limit "just" up to 1000
> datasets.
>
> Well, one option would be to use the GDALProxyPoolDataset mechanism, but this
> is a rather advanced/internal/undocumented feature used by the VRT driver to
> be able to deal with an arbitrary number of sources. If you badly need it, you
> may study gcore/gdal_proxy.h and look how it used by frmts/vrt/vrtsources.cpp.
>
> Even
>
>> "ERROR 4: Failed to re-open
>> /vsizip//mnt/data1/download_asar/ASA_WSM_1PNPDE20110731_205400_000002143105
>> _00100_49252_3499.zip/ASA_WSM_1PNPDE20110731_205400_000002143105_00100_4925
>> 2_3499.data/Sigma0_VV.img within ENVI driver."
>> I am running gdal 1.9.0 on a Debian Linux from within a c++ Programm
>>
>> Is there any way to increase this Limit? Preferably without having to
>> recompile gdal, as i do not have Admin rights on the servers it runs on?
>>
>> Thanks
>> Julian
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev