[gdal-dev] How to handle private (system) tables which a caller may explicitly want

[Nyall Dawson]

On Sun, 22 Aug 2021 at 18:59, Even Rouault <even.rouault at spatialys.com> wrote:
>
> Hi Nyall,
>
> There are several possible approaches to this:
>
> - have a LIST_ALL_TABLES=YES open option. A few drivers expose it, like
> PG, SQLite and GPKG. But the intent here was more for drivers that use
> by default the content of a geometry_columns table to establish their
> list of layers. With LIST_ALL_TABLES=YES they expose also other
> non-spatial tables. But if that includes "system" tables or not is
> driver dependent. I see the GPKG driver still hides system tables,
> whereas SQLite really shows all tables, and PG exposes for example
> raster tables (but not pg_XXXX tables, but I don't think they are actual
> tables, so that might explain it)
>

I like this approach. Would you be accepting of a new API "bool
GDALDataset::IsLayerPrivate(int iLayer)" to accompany this? It would
return true for any layers which are deemed as private/system/internal
details only.

Nyall



> - have a GetLayerByName() implementation that creates on-demand layers.
> This is implement, among other, for example in the SQLite driver. In
> that case, those new layers must be added to the layer list (ie freeing
> them is done by the driver at dataset closing)
>
> - or just rely on ExecuteSQL("SELECT * FROM the_table"). This generally
> works for most drivers without adding new code, and from a quick test,
> this works with the PGeo driver. Except that MDBTools emits errors on
> the console when trying on a non-existing table. I guess some quick
> processing of the SQL string could be done to extract the table name,
> check if it exists and it it doesn't just emits a OGR error. Or have a
> special handling of a "SELECT * FROM "_ogr_table_list_" " that would
> return them ?
>
> Even
>
> Le 22/08/2021 à 06:33, Nyall Dawson a écrit :
> > Hi list,
> >
> > Looking for some advice here on the best way to handle a particular
> > situation in some particular GDAL formats, specifically formats which
> > have a bunch of internal or system tables which we don't want to
> > expose to users in 99.99999% of cases BUT still allowing the 0.000001%
> > to explicitly get access to the tables when needed.
> >
> > An example: the ESRI personal geodatabase format has a bunch of tables
> > like "GDB_Annosymbols", "GDB_Relationships", "GDB_TopoRules", etc. In
> > almost all cases a user won't want to see these tables exposed from
> > client applications so they are currently hardcoded to be skipped over
> > when building the list of layers for a datasource.  BUT there's
> > actually some useful content in these tables which an application
> > might want to read in very specific cases, e.g. if the application
> > wanted to read the topo rules to convert these to a human readable
> > form, or the relationship information, etc.
> >
> > I can't see any way to handle this in current GDAL api -- either a
> > layer is exposed and accessible or it's skipped and there's no way for
> > a client to access it. Am I missing something, or would this require
> > new API? If so, has anyone any thoughts on how these could be handled
> > nicely?
> >
> > Nyall
> > _______________________________________________
> > gdal-dev mailing list
> > gdal-dev at lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> --
> http://www.spatialys.com
> My software is free, but my time generally not.
>