[gdal-dev] Check of sibling files when opening an image file

[Armin Burger]

Just in case someone is interested in a solution for this issue:

The most efficient setting was to set the gdal option
   GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR

This can reduce the file stat/open calls noticeably, depending on the
raster format. The worst format I found was JP2, which is now
unfortunately more often used like e.g. in Sentinel-2 data. For a JP2
image, a gdal Open() request for a single JP2 file makes 46 file
stat/open calls... With the above setting this is reduced to 6. In order
to set this as default, one needs to change this in the source code in
file gcore/gdalopeninfo.cpp.

If in addition the option GDAL_PAM_ENABLED is set to NO, then another 5
stat/open calls are eliminated. Here this setting can be put as default
at compile time by adding to the "configure" script the option
   --without-pam

Cheers
Armin




On 21/03/2019 19:54, Armin Burger wrote:
>
>
> On 21/03/2019 19:09, Even Rouault wrote:
>> On jeudi 21 mars 2019 18:34:11 CET Armin Burger wrote:
>>> Hi all
>>>
>>> When gdal opens a file, it checks for various sibling files, like *.MTL,
>>> *.aux.xml, *.RPC, etc (this is the case when the full directory scanning
>>> is deactivated). This can be ~30 stat or open requests when opening a
>>> single image file. Is there a way to define the file types that gdal
>>> checks for their existence, thus allowing to reduce this to the pure
>>> minimum, like *.ovr files only? Is this check driver dependent?
>>
>> If using /vsicurl/ and derived file systems, you can use
>> CPL_VSIL_CURL_ALLOWED_EXTENSIONS=".tif,.ovr" for example
>>
>> Even
>>
>
> Hi Even
>
> Thanks, but we're using a POSIX file system (mounted via FUSE client),
> so I assume this will not work then.
>
> It would also be fine for us to reduce the number of extensions at
> compile time by modifying configuration or source code files. But so far
> I haven't found any appearances in the source code files for file
> extensions that are obviously by default checked for their existence,
> like e.g. "RPC.TXT". So any hints if/how this is possible would be great.
>
> Cheers
> Armin
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev