[gdal-dev] Ignore content-length in vsicurl?

[Daniel Evans]

Hi all,

I am attempting to read a dataset via /vsicurl/ where I believe the server
is incorrectly returning `content-length: 0` in response to HEAD requests.
This causes GDAL to believe it's a zero-length file, and it therefore can't
be read.

If I download the file via HTTP GET, it's valid, and GDAL can read it
locally. I've also confirmed I can use /vsicurl/ on some test datasets in
the GDAL repo.

Is it possible to force GDAL to work around the faulty content-length
header, or is it too fundamental a problem to ignore?

I've separately got in touch with the data provider to see if they are able
to fix the issue at their end.

Cheers,
Daniel


URL of the troublesome dataset:
https://data.source.coop/earthgenome/sentinel2-temporal-mosaics/20NMH_2024-04-01_2024-08-01/B08.tif


Example HTTP header responses I'm seeing:

GET

HTTP/2 200
date: Tue, 10 Sep 2024 13:47:54 GMT
content-type: binary/octet-stream
content-length: 278198294
vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
etag: "a79f3f685281d6681e4d362536c5b3eb-34"
last-modified: Thu, 25 Jul 2024 13:16:08 GMT
x-version: 0.0.16
access-control-allow-credentials: true

HEAD

HTTP/2 200
date: Tue, 10 Sep 2024 13:48:08 GMT
content-type: binary/octet-stream
content-length: 0
x-version: 0.0.16
access-control-allow-credentials: true
etag: "a79f3f685281d6681e4d362536c5b3eb-34"
last-modified: Thu, 25 Jul 2024 13:16:08 GMT
vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240910/db9f9701/attachment.htm>