[gdal-dev] Virtual Raster Tile Index (VRTTI) driver, and associated gdaltindex improvements

[Even Rouault]

Sean,
>
> I'm wondering how this relates to STAC. Do you imagine that GDAL users 
> should and will use this to publish large collections of data? When 
> should they use this and when should they use STAC collections instead?
I don't imagine this for publishing (that is not publishing the VRRTI 
tile index itself), but more for internal use in the software stack of 
users with large image collections. Similarly to good-ol' XML VRT.
>
> Back in the day our tile indexes were shapefiles. I don't think this 
> would be a good practice now. Would all vector formats be supported?
yes, in theory. But as documented, it is strongly recommended to use 
vector formats with efficient spatial filtering like GeoPackage, 
FlatGeoBuf, PostGIS. Those 3 ones in particular have been enhanced to 
support setting the metadata items used by VRTTI driver. So you don't 
necessarily need to create a XML VRTTI wrapper file for those.
> How about column-oriented formats?
that should work with them too. The fact that they are column-oriented 
is just an implementation detail. That said, until GeoParquet gets 
efficient spatial filtering (in the works at the spec level: 
https://github.com/opengeospatial/geoparquet/pull/191), this isn't yet a 
good choice (as a provision the VRTTI driver actually already supports 
.vrt.parquet)
>  Can I just do a gdalinfo "XXXX:example.shp" on a MapServer tile index?
Yes. However, given that the metadata items suggested by VRTTI will not 
be available, this will require the driver to GDALOpen() one of the tile 
to figure out the number of bands, data type, etc. at opening time. Not 
a big deal however if the VRTTI dataset is opened for a long time to 
satisfy many requests.
>
> This feature has been latent and available for a while, yes? It's 
> obviously useful. My main misgivings are about the overlap with 
> existing GDAL features and protocols (VRT, WMS, WMTS, STAC, MBTiles). 
> I think too many choices can create a not great experience for users.

Yes there is clearly overlap (like most GDAL supported formats have 
strong functional overlap since they can be used with a common API :-))

Here are the main differences I see:

- WMS: requires a client&server architecture

- WMTS: generally requires a client&server architecture too (local use 
possible if using file:// URIs), but requires regular tiling

- STAC(ITems, with STAC projection extensions): functionally a bit 
similar to VRTTI, but using JSON format to catalog the sources. If you 
want to manage a catalog with a large set of sources, static STAC will 
not work well and you'll need a STAC server. Although with STAC, you 
want each "asset" to have user-facing metadata. In a VRTTI use case, you 
are more interested by the resulting mosaic than each of its source.

- STAC(Temporal Asset): functionally similar to WMTS (regular tiling)

- MBTiles/GeoPackage/COG/etc.: single-file tiled formats

Even

>
> On Wed, Dec 20, 2023, 11:33 AM Even Rouault via gdal-dev 
> <gdal-dev at lists.osgeo.org> wrote:
>
>     Hi,
>
>     For those not actively following github tickets & PR, I just want to
>     point to a new pending major functionality to improve management of
>     virtual mosaics with a very large number of tiles/sources (> tens of
>     thousands of tiles), by referencing them as features of a vector
>     layer
>     (typically created by gdaltindex), instead of a XML file as in
>     traditional VRT, augmented with additional metadata.
>
>     More details in https://github.com/OSGeo/gdal/pull/8983 (and in
>     initial
>     ticket in https://github.com/OSGeo/gdal/issues/8861)
>
>     Even
>
-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20231221/c8be3c0a/attachment.htm>