[gdal-dev] Passing open options along dataset name in a string ?

Tue Nov 17 01:57:29 PST 2020

Hi,

I have done some helpdesk work within the GDAL community and I know well
that the open options and config options are confusing. I also know that
they exists for a reason but simplified and uniform way to use them would be
nice.

Some comments on comments:

>> gdalinfo my.tif -oo GEOREF_SOURCES=WORLDFILE,PAM
>>

> Ideally this would be baked into the format, but, yes, I think we've got a
> bead on dataset open options.

I don't know how it could be baked into the format. The option gives user an
option to override wrong GeoTIFF georeferencing with wordfile, for example.

>> gdalinfo BAG:"data/test_vr.bag":supergrid:0:1
>>

> DRIVER:"file":something

> Right. This will require some work because of multiple colons. Though I've
> never seen BAG driver data in the wild. Is this a real live format?

As far as I know BAG is the hdf of bathymetry and widely used in that
context.

>> gdalinfo data/test_vr.bag -oo MODE=RESAMPLED_GRID -oo SUPERGRIDS_MASK=YES
>> gdalinfo HDF5:"d:\foo.he5"://HDFEOS/SWATHS/foo/bar
>>

> HDF5 driver, filename using Windows drive, and UNC path within it. This is
> marginal, right?

The part beginning with // is not UNC path but the name of the subdataset
within hdf5 file https://gdal.org/drivers/raster/hdf5.html. Not more
marginal than HDF5 itself.

>> ogrinfo OCI:warmerda/password at .dreadfest

>Wat?

Text has just been formatted into email link because of the @ sign that
belongs to the Oracle connection string "username" / "password" @ "the name
of the Oracle database as it appears in the tnsnames.ora file". Let's see if
formatting happens again when I send this from Nabble:

OCI:warmerda/password at gdal800.dreadfest.com abc.shp

-Jukka Rahkonen-

Sean Gillies-3 wrote
> Hi Even,
> 
> On Wed, Nov 4, 2020 at 9:01 AM Even Rouault <

> even.rouault@

> >
> wrote:
> 
>> > > Another particularity we have in GDAL is that the dataset name might
>> be
>> > > almost
>> > > anything. Most of the time, it is a regular file path, or some /vsi
>> path.
>> > > But
>> > > sometimes, it can be JSON content (the GeoJSON driver accepts the
>> content
>> > > to
>> > > be directly provided as the dataset name), or XML (VRT, WMS drivers).
>> > > We have also the subdataset syntax "HDF5:foo.hdf:my_variable"
>> >
>> > Could VRT XML and JSON be exempted? We already have a way to embed open
>> > options in the XML.
>>
>> If the gdn: mechanism is a new possibility offered that doesn't exclude
>> existing ones (otherwise that would be a pretty big breaking change), we
>> could
>> possibly exempt the odd cases I mentioned (or have some quoting/escaping
>> rules
>> to enable that payload to be seen as a file), which generally don't need
>> a
>> "permanent" way of refering to the dataset like gdn: would offer, since
>> this
>> is content often generated programatically or retrieved dynamically.
>>
>> Covering subdataset would be a more important use case. Something that
>> would
>> have to be decided if the way we express subdatasets would be somehow
>> standardized or if it would be a black-box string for the gdn:
>> encapsulation.
>> For a black-box approach, we would have to define some escaping/quoting
>> rules
>> to avoid any potential issue with separators of the gdn syntax. If we
>> decide
>> that the subdataset syntax is part of what is standardized by GDN that
>> would
>> be a more challenging exercice, because the subdataset syntax varies from
>> driver to driver.
>>
> 
> The variation of subdataset syntax among drivers is a bug, let's try to
> fix
> this.
> 
> It seems to me that the internet way to address subdatasets would be to
> use
> a # URL fragment. But since most of our formats and the servers that serve
> files of these formats are not aware, we may have to come up with
> something
> different. We may need to consider making subdatasets a layer opening
> option?
> 
> pending on how we design things, that might impact between:
>> - just GDALOpen() generic code if GDALOpen() decodes the gdn: string to
>> decompose it into 'classic' dataset names and open options
>> - all drivers if the gdn: string would be passed to each
>> GDALDriver::pfnOpen()
>> implementation
>> - intermediate situation if we decide to drop (at least for future
>> drivers)
>> per-driver subdataset syntax (which has deficiencies has the quoting
>> rules
>> to
>> separate the filename from the non-filename component vary from driver to
>> driver, and are most of the time not defined) to come up with something
>> more
>> standardized
>>
>> To help brainstorming, a non-exhaustive overview of a few situations
>> mixing
>> driver prefixing, subdataset syntax and open options:
>>
>> gdalinfo my.tif
>>
> 
> Yes. We have to handle bare paths to local dataset files.
> 
> 
>> gdalinfo my.tif -oo GEOREF_SOURCES=WORLDFILE,PAM
>>
> 
> Ideally this would be baked into the format, but, yes, I think we've got a
> bead on dataset open options.
> 
> 
>> gdalinfo GTIFF_DIR:0:d:\my.tif
>>
> 
> WTF is this? :)
> 
> 
>> gdalinfo EEDAI:my/asset
>> gdalinfo EEDAI: -oo ASSET=my/asset
>> gdalinfo EEDAI:my/asset:band1, band2
>> gdalinfo EEDAI: -oo ASSET=my/asset -oo BANDS=band1,band2
>>
> 
> Never seen these.
> 
> 
>> gdalinfo BAG:"data/test_vr.bag":supergrid:0:1
>>
> 
> DRIVER:"file":something
> 
> Right. This will require some work because of multiple colons. Though I've
> never seen BAG driver data in the wild. Is this a real live format?
> 
> 
>> gdalinfo data/test_vr.bag -oo MODE=RESAMPLED_GRID -oo SUPERGRIDS_MASK=YES
>> gdalinfo HDF5:"d:\foo.he5"://HDFEOS/SWATHS/foo/bar
>>
> 
> HDF5 driver, filename using Windows drive, and UNC path within it. This is
> marginal, right?
> 
> 
>> gdalinfo netCDF:"/vsicurl/http://example.com/my.nc":my_var
>>
> 
> This looks less complicated than some of the examples above.
> 
> 
>> ogrinfo "PG:dbname=testdb user=foo"
>> ogrinfo "mySQL:testdb,user=foo"
>>
> 
> These seem like they could be driver specific, but generalized key-value
> parameters.
> 
> 
>> ogrinfo OCI:warmerda/

> password at .dreadfest

> 
> 
> Wat?
> 
> 
>> GDALOpen() is not even aware that HDF5:bla means that the dataset will be
>> recognized by the HDF5 driver
>>
>>
> Wait what?
> 
> -- 
> Sean Gillies
> 
> _______________________________________________
> gdal-dev mailing list

> gdal-dev at .osgeo

> https://lists.osgeo.org/mailman/listinfo/gdal-dev

--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html