[gdal-dev] RANGE Requests on large Rasters with CDN (AWS Cloudfront)

[Sean Turner]

Hi all,

Curious if anyone is doing something similar.

We've got our data in an S3 Bucket that is fronted by Cloudfront. From
there we can use the GDAL `GDAL_HTTP_HEADER_FILE` environment variable to
point to a file containing Cloudfront signed cookies (which control access
to files within Cloudfront).

Cloudfront supports a maximum file size of 50gb. Trying to add a raster
layer that is greater than 50gb immediately results in an error.

Is anyone doing something similar and have a work around? AWS Documentation
mentions RANGE requests where a range of incremental 50gb are requested
from Cloudfront and downloaded simultaneously. Is this supported by GDAL
(QGIS)?

Otherwise also considering using a VRT?
https://gdal.org/en/latest/drivers/raster/vrt.html

Open to any and all suggestions!

Cheers. Also, sorry I haven't done one of these mailing lists before,
apologies if this submission is in the wrong place.

All the best,
Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240909/8778524f/attachment.htm>