[gdal-dev] Passing open options along dataset name in a string ?

[Sean Gillies]

Hi Even,

On Wed, Nov 4, 2020 at 9:01 AM Even Rouault <even.rouault at spatialys.com>
wrote:

> > > Another particularity we have in GDAL is that the dataset name might be
> > > almost
> > > anything. Most of the time, it is a regular file path, or some /vsi
> path.
> > > But
> > > sometimes, it can be JSON content (the GeoJSON driver accepts the
> content
> > > to
> > > be directly provided as the dataset name), or XML (VRT, WMS drivers).
> > > We have also the subdataset syntax "HDF5:foo.hdf:my_variable"
> >
> > Could VRT XML and JSON be exempted? We already have a way to embed open
> > options in the XML.
>
> If the gdn: mechanism is a new possibility offered that doesn't exclude
> existing ones (otherwise that would be a pretty big breaking change), we
> could
> possibly exempt the odd cases I mentioned (or have some quoting/escaping
> rules
> to enable that payload to be seen as a file), which generally don't need a
> "permanent" way of refering to the dataset like gdn: would offer, since
> this
> is content often generated programatically or retrieved dynamically.
>
> Covering subdataset would be a more important use case. Something that
> would
> have to be decided if the way we express subdatasets would be somehow
> standardized or if it would be a black-box string for the gdn:
> encapsulation.
> For a black-box approach, we would have to define some escaping/quoting
> rules
> to avoid any potential issue with separators of the gdn syntax. If we
> decide
> that the subdataset syntax is part of what is standardized by GDN that
> would
> be a more challenging exercice, because the subdataset syntax varies from
> driver to driver.
>

The variation of subdataset syntax among drivers is a bug, let's try to fix
this.

It seems to me that the internet way to address subdatasets would be to use
a # URL fragment. But since most of our formats and the servers that serve
files of these formats are not aware, we may have to come up with something
different. We may need to consider making subdatasets a layer opening
option?

pending on how we design things, that might impact between:
> - just GDALOpen() generic code if GDALOpen() decodes the gdn: string to
> decompose it into 'classic' dataset names and open options
> - all drivers if the gdn: string would be passed to each
> GDALDriver::pfnOpen()
> implementation
> - intermediate situation if we decide to drop (at least for future
> drivers)
> per-driver subdataset syntax (which has deficiencies has the quoting rules
> to
> separate the filename from the non-filename component vary from driver to
> driver, and are most of the time not defined) to come up with something
> more
> standardized
>
> To help brainstorming, a non-exhaustive overview of a few situations
> mixing
> driver prefixing, subdataset syntax and open options:
>
> gdalinfo my.tif
>

Yes. We have to handle bare paths to local dataset files.


> gdalinfo my.tif -oo GEOREF_SOURCES=WORLDFILE,PAM
>

Ideally this would be baked into the format, but, yes, I think we've got a
bead on dataset open options.


> gdalinfo GTIFF_DIR:0:d:\my.tif
>

WTF is this? :)


> gdalinfo EEDAI:my/asset
> gdalinfo EEDAI: -oo ASSET=my/asset
> gdalinfo EEDAI:my/asset:band1, band2
> gdalinfo EEDAI: -oo ASSET=my/asset -oo BANDS=band1,band2
>

Never seen these.


> gdalinfo BAG:"data/test_vr.bag":supergrid:0:1
>

DRIVER:"file":something

Right. This will require some work because of multiple colons. Though I've
never seen BAG driver data in the wild. Is this a real live format?


> gdalinfo data/test_vr.bag -oo MODE=RESAMPLED_GRID -oo SUPERGRIDS_MASK=YES
> gdalinfo HDF5:"d:\foo.he5"://HDFEOS/SWATHS/foo/bar
>

HDF5 driver, filename using Windows drive, and UNC path within it. This is
marginal, right?


> gdalinfo netCDF:"/vsicurl/http://example.com/my.nc":my_var
>

This looks less complicated than some of the examples above.


> ogrinfo "PG:dbname=testdb user=foo"
> ogrinfo "mySQL:testdb,user=foo"
>

These seem like they could be driver specific, but generalized key-value
parameters.


> ogrinfo OCI:warmerda/password at gdal800.dreadfest.com


Wat?


> GDALOpen() is not even aware that HDF5:bla means that the dataset will be
> recognized by the HDF5 driver
>
>
Wait what?

-- 
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20201116/71b3c4e4/attachment.html>