[gdal-dev] VSI sidecar and sibling file lookup

[Even Rouault]

Sean,

did you implement the Stat() method ? From gdalpamdataset.cpp lines 921 
to 940, for .aux.xml to be used, you either need to implement 
ReadDir()/VSISiblingFiles or Stat(). Actually, I'd strongly recommend 
implementing Stat() as a lot of places will assume it to be working. As 
you can skip establishing the directory listing with 
GDAL_DISABLE_READDIR_ON_OPEN, all code should be robust to an empty 
readdir() result, but it will then assume Stat() to work.

The .aux file is a different story, since it is actually attempted to be 
opened with GDALOpen(), which first tries a Open(), and doesn't require 
Stat() to succeed.

.ovr files are only attempted to be opened if you do a RasterIO() call 
that involves subsampling.  A plain GDALOpen() on a .tif doesn't involve 
sidecar file loading. The GTiff reader implements a on-demand logic to 
try to open them

Even

Le 17/02/2024 à 00:00, Sean Gillies via gdal-dev a écrit :
> Update... It looks like there are at least two different kinds of 
> logic around sidecar/sibling files.
>
> My Rasterio VSI plugin currently implements only open, tell, seek, 
> read, write, and close. The mandatory methods. I can see GDAL probing 
> for .aux and .AUX files. But not .aux.xml files for some reason that I 
> don't understand. The dataset I'm exposing through this VSI plugin has 
> a .ovr file, but there's no attempt to find it. It looks like .aux.xml 
> and .ovr require VSI sibling files and/or readdir support, but that 
> .aux file support does not? I'm deducing this from the behavior of the 
> system. It would be hard to figure it out just by reading the code.
>
> On Fri, Feb 16, 2024 at 8:29 AM Sean Gillies <sean.gillies at gmail.com> 
> wrote:
>
>     Thanks for the tip, Thomas!
>
>     On Thu, Feb 15, 2024 at 9:45 AM thomas bonfort
>     <thomas.bonfort at gmail.com> wrote:
>
>         Hi Sean,
>         I believe/recall this is very driver dependent. Some drivers
>         will look for their sidecars in the provided sibling files
>         list returned by VSISiblingFiles, whereas others will
>         unconditionally try to open well-known sidecar names they will
>         have computed from their own name. IIRC I added
>         VSISiblingFiles so that some vsi backends (namely object
>         storage based ones) could explicitly return an empty list of
>         sibling files in order to prevent drivers to emit a subsequent
>         Readdir() (which might break drivers that require sidecar
>         files to work, but speeds up those where sidecar files are
>         optional and not used in a cloud-storage environment (namely
>         cogs) )
>
>         On Tue, Feb 13, 2024 at 6:12 PM Sean Gillies via gdal-dev
>         <gdal-dev at lists.osgeo.org> wrote:
>
>             Hi all,
>
>             It's not clear to me from reading
>             https://gdal.org/user/virtual_file_systems.html if VSI
>             sidecar and sibling file lookup works in general, by
>             design, or whether it's an implementation detail of the
>             standard VSI filesystems ("vsizip", "vsicurl", etc.).
>
>             More specifically: if I have a "/vsipyopener/foo.tif"
>             object, do drivers always look for other files at paths
>             precisely relative to that one? Does this vary among drivers?
>
>
>
>     -- 
>     Sean Gillies
>
>
>
> -- 
> Sean Gillies
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240218/8dd88bdf/attachment.htm>