[gdal-dev] Indexing lots of remote COGs for use with gdal raster clip and QGIS

[Scott]

Hey Robin,

I would look into the GTI raster driver:
https://gdal.org/en/latest/programs/gdal_driver_gti_create.html#gdal-driver-gti-create

It creates a GPKG index of rasters for efficient access. Creating a GTI:

gdal driver gti create --resolution .0029,.0029 --output-layer dem30 
ned10/cogs cogs.gti.gpkg

Using the new index:
gdal raster clip -i cogs.gti.gpkg --bbox -111,36,-110,37 --of COG -o 
newCog.tif

QGIS will read the index as a vector file (since it's GPKG) displaying 
bounding boxes for each raster in the index, but not as a raster as a whole.

Hope that helps!


On 9/1/26 06:33, Robin Wilson via gdal-dev wrote:
> Hi,
> 
> I have a lot of COGs hosted online in HTTP RANGE compatible storage. I 
> want to create a catalog/index of them so that GDAL and tools that use 
> GDAL (like QGIS) can extract the relevant data from the COGs for 
> specific areas. For example, I’d like to use `gdal raster clip` to clip 
> this big ‘virtual raster’ to a specific bbox or polygon. Ideally I’d 
> also like to be able to view the big raster on-the-fly in QGIS, but I’m 
> aware that may be more difficult. My tiles are ’true tiles’ in that they 
> tessellate properly and each location is in a single file.
> 
> I’ve been looking at the GDAL Tile Index format, but have been getting a 
> bit confused, and some things haven’t been working quite how I expect. A 
> few questions below:
> 
> 1. Is there an easy way to generate a GDAL Tile Index file from a file 
> containing a list of remote URLs? I couldn’t find the right way to 
> specify a file containing URLs on the command line, so I ended up 
> writing a short Python script that loads the list of URLs from a file 
> and then passes them to gdal.TileIndex. Is that the best way to do this? 
> One disadvantage of this method is that I can’t seem to get a progress 
> bar to display.
> 
> 2. My current way of doing that requires GDAL loading the bbox of each 
> file by requesting it over the network, which is fairly slow, and I have 
> thousands of files to do this for. Is there a way to get the GTI 
> creation to use local files on my hard disk for getting the extent of 
> each file, but save the location as a /vsicurl/ URL to the remote file? 
> I’ve found --base-url in `gdal raster index` though I’m slightly 
> confused by the description of it ("Base URL for STAC-GeoParquet the 
> href property of the id property in a STAC-GeoParquet index.” - does 
> that do what I want and put a base URL in front of each file it finds?). 
> That option doesn’t seem to be present in `gdal driver gti create` 
> though, and seems like it may be STAC-GeoParquet specific.
> 
> 3. Once I’ve created the file, I can do `gdal raster clip` operations on 
> the command-line quite quickly, but if I load the GTI raster into QGIS 
> then QGIS just hangs. I hoped it would just request the relevant COGs 
> for the area I was looking at. I’ve tried loading it in by dragging and 
> dropping the .gti.gpkg file and then selecting the raster in the layers 
> popup, and also adding it via Add Layer->Raster and putting 
> ‘GTI:<path>’. I have a web map working which loads these COG files on 
> the fly depending on where you’re looking (using a minimal STAC catalog 
> in a JSON file) and was hoping to have the same thing working in QGIS, 
> without the effort of setting up something like titiler or titiler- 
> pgstac, as I’m currently managing to get away without running any 
> servers  (as opposed to static data stores) for this data.
> 
> If there’s an alternative method I should be looking at for doing either 
> part of this (the clipping or the viewing in QGIS) then let me know - 
> I’m aware I may be pushing the boundaries of what can be sensibly done 
> with the GTI format.
> 
> Best regards,
> 
> Robin
> 
> 
> Dr Robin Wilson
> 
> 
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev