[gdal-dev] [EXTERNAL] s3 request example

Fri Sep 13 09:14:36 PDT 2024

Thanks David for the suggestion. Very interesting data.

Unfortunately the issue I have is not reproduced with that file. So it may
be due to the authentication (your suggestion does not need authentication).

Some context: I am opening a COG file from S3 in QGIS, setting the url in
the command line. Something like
qgis /vsis3/blahblabla

(more on that later)

If I run it in my Ubuntu 22.04 (that has GDAL 3.4.1), with QGIS 3.38, no
problem
If I run with conda with QGIS 3.38 (that has GDAL 3.9.1), it is noticeable
slower.

So I think it is due to the version of GDAL. If you think that I am wrong
and I should ask directly to QGIS, it is ok.

This is a bit problematic to reproduce because the authorization expires in
12 hours. So I post here the command I produced with a python script. If it
is needed, I can share the python script that is  getting the
authentication. It is about 100 lines of code.

Enabling the logs I see there a gap of 6 seconds during the opening with
conda, that is not there with the QGIS from Ubuntu that uses GDAL 3.4.1

This is the log from conda
The gap is between
Fri Sep 13 17:56:26 2024].9370
and
[Fri Sep 13 17:56:33 2024].4445

CPL_DEBUG=ON CPL_TIMESTAMP=ON python3 pix4d_qgis.py --project-id 1969386
--share-token 09c3eada-1b3b-46eb-b5de-8bd51635a5b9
Session expires at 2024-09-14T03:56:22Z
AWS_ACCESS_KEY_ID=ASIATOCJLBKS2RHGCAHB
AWS_SECRET_ACCESS_KEY=nNT8fTH/cGiyn4NVubp5/1LroyZ69johhXYxkCK1
AWS_SESSION_TOKEN=FwoGZXIvYXdzEKn//////////wEaDI1G/49ov88aGibrOCLXAilDbBuFQ/ydJF4AZ50/lYmWYmNDjM9H7XbbioyXiLQzwTqVRN1TLG6T29Anklkw47MKxdKHDF/O85r9gWJmCAqYTKalA38VM1qbixsjSXAzR7W5p1v1pgRUKtN4J03NrZOH2S6gbo4OKv+UUkPe/Xq4s8w0vEVNJLhvVx6dvpJMv7Jp9UxvLA5a+nrBKUqJujujqJnRXAKKfnpJbTDvhCuIeTxaFFt50e5UpLjSwqXS/a/AaEMJXRSsSLbKFTW/ALSCHfPv3HCStnue5CFpzVk4OhQbQ+dZyXUQp5wxQTtpr0hIh0IyB20gsoZNa0lGvFuXnzqDCH2X2mXwzDcyZk2IOX34nq+Wtnk8qYdjJxEAG0tnlYsMjmmXECnbOxl6gtNJ5hoehgak6cSfs7/rMi4jOI838tzsWWyDJeACj9qfQyfGtpk23x8Zt6KuqUjNZ1/z+/T8LwoopsGRtwYyKQbJS8pn8IGKPLtHDVxRx09yBRuqgYVFzSqpnr3T1vwWYHXiEF0sVJBx
GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
/home/jshaw/anaconda3/envs/qgis_stable/bin/qgis
/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff
[Fri Sep 13 17:56:26 2024].3948, 0.0000: HTTP: libcurl/8.9.0 OpenSSL/3.3.1
zlib/1.3.1 zstd/1.5.6 libssh2/1.11.0 nghttp2/1.58.0
[Fri Sep 13 17:56:26 2024].7731, 0.3783: S3: Switching to endpoint
s3.eu-central-1.amazonaws.com
[Fri Sep 13 17:56:26 2024].8415, 0.4467: S3: Switching to region
eu-central-1
[Fri Sep 13 17:56:26 2024].9230, 0.5282: S3: GetFileSize(
https://prod-pix4d-cloud-eu-central-1.s3.eu-central-1.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff)=66132291
 response_code=206
[Fri Sep 13 17:56:26 2024].9235, 0.5288: GDAL:
GDALOpen(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x556e6bb915a0) succeeds as GTiff.
[Fri Sep 13 17:56:26 2024].9240, 0.5292: GTiff: ScanDirectories()
[Fri Sep 13 17:56:26 2024].9243, 0.5296: GTiff: Opened band mask.
[Fri Sep 13 17:56:26 2024].9246, 0.5298: GTiff: Opened 8929x7207 overview.
[Fri Sep 13 17:56:26 2024].9249, 0.5301: GTiff: Opened 4464x3603 overview.
[Fri Sep 13 17:56:26 2024].9251, 0.5304: GTiff: Opened 2232x1801 overview.
[Fri Sep 13 17:56:26 2024].9254, 0.5306: GTiff: Opened 1116x900 overview.
[Fri Sep 13 17:56:26 2024].9257, 0.5309: GTiff: Opened 558x450 overview.
[Fri Sep 13 17:56:26 2024].9259, 0.5311: GTiff: Opened 279x225 overview.
[Fri Sep 13 17:56:26 2024].9262, 0.5314: GTiff: Opened band mask for
8929x7207 overview.
[Fri Sep 13 17:56:26 2024].9264, 0.5316: GTiff: Opened band mask for
4464x3603 overview.
[Fri Sep 13 17:56:26 2024].9267, 0.5319: GTiff: Opened band mask for
2232x1801 overview.
[Fri Sep 13 17:56:26 2024].9269, 0.5321: GTiff: Opened band mask for
1116x900 overview.
[Fri Sep 13 17:56:26 2024].9272, 0.5324: GTiff: Opened band mask for
558x450 overview.
[Fri Sep 13 17:56:26 2024].9274, 0.5326: GTiff: Opened band mask for
279x225 overview.
[Fri Sep 13 17:56:26 2024].9277, 0.5329: GDAL:
GDALClose(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x556e6bb915a0)
ERROR: Status 2: File
/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff
could not be found
[Fri Sep 13 17:56:26 2024].9370, 0.5422: GDAL: On-demand registering
/home/jshaw/anaconda3/envs/qgis_stable/lib/gdalplugins/gdal_TileDB.so using
GDALRegister_TileDB.
[Fri Sep 13 17:56:33 2024].4445, 7.0498: S3: GetFileSize(
https://prod-pix4d-cloud-eu-central-1.s3.eu-central-1.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff)=66132291
 response_code=206
[Fri Sep 13 17:56:33 2024].4449, 7.0501: GDAL:
GDALOpen(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x556e6b807d00) succeeds as GTiff.
GDAL ERROR 0: [Fri Sep 13 17:56:33 2024].4666, 7.0718: OGRCT: Using
WebMercator to WGS84 optimization
GDAL ERROR 0: [Fri Sep 13 17:56:33 2024].4667, 7.0719: GDAL: Computing area
of interest: 6.62976, 46.6391, 6.63534, 46.6422
[Fri Sep 13 17:56:33 2024].4667, 7.0719: GTiff: ScanDirectories()
[Fri Sep 13 17:56:33 2024].4669, 7.0721: GTiff: Opened band mask.
[Fri Sep 13 17:56:33 2024].4670, 7.0722: GTiff: Opened 8929x7207 overview.
[Fri Sep 13 17:56:33 2024].4671, 7.0723: GTiff: Opened 4464x3603 overview.
[Fri Sep 13 17:56:33 2024].4672, 7.0724: GTiff: Opened 2232x1801 overview.
[Fri Sep 13 17:56:33 2024].4673, 7.0725: GTiff: Opened 1116x900 overview.
[Fri Sep 13 17:56:33 2024].4674, 7.0726: GTiff: Opened 558x450 overview.
[Fri Sep 13 17:56:33 2024].4675, 7.0727: GTiff: Opened 279x225 overview.
[Fri Sep 13 17:56:33 2024].4676, 7.0728: GTiff: Opened band mask for
8929x7207 overview.
[Fri Sep 13 17:56:33 2024].4677, 7.0729: GTiff: Opened band mask for
4464x3603 overview.
[Fri Sep 13 17:56:33 2024].4677, 7.0730: GTiff: Opened band mask for
2232x1801 overview.
[Fri Sep 13 17:56:33 2024].4678, 7.0731: GTiff: Opened band mask for
1116x900 overview.
[Fri Sep 13 17:56:33 2024].4679, 7.0732: GTiff: Opened band mask for
558x450 overview.
[Fri Sep 13 17:56:33 2024].4680, 7.0733: GTiff: Opened band mask for
279x225 overview.
[Fri Sep 13 17:56:33 2024].5178, 7.1230: GDAL:
GDALOpen(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x7f830c08d490) succeeds as GTiff.
[Fri Sep 13 17:56:33 2024].5180, 7.1232: S3: Downloading 59179008-59244543 (
https://prod-pix4d-cloud-eu-central-1.s3.eu-central-1.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
GDAL ERROR 0: [Fri Sep 13 17:56:33 2024].5244, 7.1296: GDAL: Computing area
of interest: 6.62976, 46.6391, 6.63534, 46.6422
[Fri Sep 13 17:56:33 2024].5245, 7.1297: GTiff: ScanDirectories()
[Fri Sep 13 17:56:33 2024].5246, 7.1298: GTiff: Opened band mask.
[Fri Sep 13 17:56:33 2024].5247, 7.1299: GTiff: Opened 8929x7207 overview.
[Fri Sep 13 17:56:33 2024].5247, 7.1300: GTiff: Opened 4464x3603 overview.
[Fri Sep 13 17:56:33 2024].5248, 7.1300: GTiff: Opened 2232x1801 overview.
[Fri Sep 13 17:56:33 2024].5249, 7.1301: GTiff: Opened 1116x900 overview.
[Fri Sep 13 17:56:33 2024].5249, 7.1302: GTiff: Opened 558x450 overview.
[Fri Sep 13 17:56:33 2024].5250, 7.1302: GTiff: Opened 279x225 overview.
[Fri Sep 13 17:56:33 2024].5251, 7.1303: GTiff: Opened band mask for
8929x7207 overview.
[Fri Sep 13 17:56:33 2024].5251, 7.1304: GTiff: Opened band mask for
4464x3603 overview.
[Fri Sep 13 17:56:33 2024].5252, 7.1304: GTiff: Opened band mask for
2232x1801 overview.
[Fri Sep 13 17:56:33 2024].5253, 7.1305: GTiff: Opened band mask for
1116x900 overview.
[Fri Sep 13 17:56:33 2024].5253, 7.1306: GTiff: Opened band mask for
558x450 overview.
[Fri Sep 13 17:56:33 2024].5254, 7.1306: GTiff: Opened band mask for
279x225 overview.
[Fri Sep 13 17:56:33 2024].5255, 7.1307: GDAL: Selecting overview 1116 x 900
[Fri Sep 13 17:56:33 2024].5255, 7.1307: S3: Downloading 16384-32767 (
https://prod-pix4d-cloud-eu-central-1.s3.eu-central-1.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 17:56:33 2024].6116, 7.2168: S3: Got response_code=206
[Fri Sep 13 17:56:33 2024].6118, 7.2171: S3: Downloading 180224-589823 (
https://prod-pix4d-cloud-eu-central-1.s3.eu-central-1.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 17:56:33 2024].6955, 7.3008: S3: Got response_code=206
[Fri Sep 13 17:56:33 2024].6960, 7.3012: GDAL: GDAL_CACHEMAX = 1591 MB
[Fri Sep 13 17:56:33 2024].6977, 7.3029: S3: Got response_code=206

This is the equivalent code with QGIS 3.38 in Ubuntu and GDAL 3.4.1

CPL_DEBUG=ON CPL_TIMESTAMP=ON python3 pix4d_qgis.py --project-id 1969386
--share-token 09c3eada-1b3b-46eb-b5de-8bd51635a5b9
Session expires at 2024-09-14T04:00:10Z
AWS_ACCESS_KEY_ID=ASIATOCJLBKSRCEUBKE6
AWS_SECRET_ACCESS_KEY=h6NhBf9VwBZoVj5Zd/8H2BnzXQu0l6bFUUv2D8E7
AWS_SESSION_TOKEN=FwoGZXIvYXdzEKn//////////wEaDF1xO50XL6GBe/358CLXAn3JR/uteJtarwJ/ryn6i4+lcMFEQZ/KV0FZQdKsD1PwAcqoYFY+0GKYSV+FTk81ZszsVzy8N7d2GwByvP9S/L1XOo1FrqJfDpGw3a5eQR7AoeU68EPlD/WTYnM7XBzXimHywDCyue3SOi8oZaSdbMgjBQ1N6j+qQ+EDN7RjD/Zu2CGMTzN9g/Z0FhZsuyt2HRhnOYpsvpUMZ+WQr7pbjdVdwt3PdFHsJ95fhueapJspgQ7OSgmj5VtSR8NTecohLsd2eeGkSpiqVWuBwuAu2cW6LcqmAZGnF0DZIjPWiMIDf/OLczMRCw5MJGeYpsSkkiuM+t9Miv8aN4iCo8vjl6wumOCgPqTgBBWlQdAs2mLxwpF1Bb1UjJ6eiccp17+yJF3ffn8tvxx/+1V3Q++E5Xk+jiB1keJnFcizr7qD6EY+EdDES5sQrWAhM2uR78JjNc0f+hVZEmAoisORtwYyKb0po07ytiiBv+X3j+Y6BsCHR0Es3C3xVvsgkfn1zb1Z2flFWUzwqJUD
GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR /usr/bin/qgis
/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff
[Fri Sep 13 18:00:14 2024].7665, 0.0000: HTTP: libcurl/7.81.0 OpenSSL/3.0.2
zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0
(+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3
OpenLDAP/2.5.18
[Fri Sep 13 18:00:14 2024].7666, 0.0001: S3: Downloading 0-16383 (
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 18:00:14 2024].8897, 0.1231: S3: Got response_code=400
[Fri Sep 13 18:00:14 2024].8898, 0.1232: S3: Switching to region
eu-central-1
[Fri Sep 13 18:00:14 2024].8899, 0.1234: S3: Downloading 0-16383 (
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 18:00:15 2024].0287, 0.2621: S3: Got response_code=206
[Fri Sep 13 18:00:15 2024].0288, 0.2622: S3: GetFileSize(
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff)=66132291
 response_code=206
[Fri Sep 13 18:00:15 2024].0292, 0.2627: GDAL:
GDALOpen(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x560e4cd84920) succeeds as GTiff.
[Fri Sep 13 18:00:15 2024].0298, 0.2632: GTiff: ScanDirectories()
[Fri Sep 13 18:00:15 2024].0301, 0.2635: GTiff: Opened band mask.
[Fri Sep 13 18:00:15 2024].0304, 0.2638: GTiff: Opened 8929x7207 overview.
[Fri Sep 13 18:00:15 2024].0306, 0.2641: GTiff: Opened 4464x3603 overview.
[Fri Sep 13 18:00:15 2024].0309, 0.2643: GTiff: Opened 2232x1801 overview.
[Fri Sep 13 18:00:15 2024].0311, 0.2646: GTiff: Opened 1116x900 overview.
[Fri Sep 13 18:00:15 2024].0314, 0.2648: GTiff: Opened 558x450 overview.
[Fri Sep 13 18:00:15 2024].0316, 0.2651: GTiff: Opened 279x225 overview.
[Fri Sep 13 18:00:15 2024].0319, 0.2654: GTiff: Opened band mask for
8929x7207 overview.
[Fri Sep 13 18:00:15 2024].0322, 0.2656: GTiff: Opened band mask for
4464x3603 overview.
[Fri Sep 13 18:00:15 2024].0324, 0.2658: GTiff: Opened band mask for
2232x1801 overview.
[Fri Sep 13 18:00:15 2024].0326, 0.2661: GTiff: Opened band mask for
1116x900 overview.
[Fri Sep 13 18:00:15 2024].0329, 0.2663: GTiff: Opened band mask for
558x450 overview.
[Fri Sep 13 18:00:15 2024].0331, 0.2665: GTiff: Opened band mask for
279x225 overview.
[Fri Sep 13 18:00:15 2024].0334, 0.2669: GDAL:
GDALClose(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x560e4cd84920)
ERROR: Status 2: File
/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff
could not be found
[Fri Sep 13 18:00:15 2024].0357, 0.2691: S3: Downloading 0-16383 (
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 18:00:15 2024].0762, 0.3096: S3: Got response_code=206
[Fri Sep 13 18:00:15 2024].0762, 0.3097: S3: GetFileSize(
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff)=66132291
 response_code=206
[Fri Sep 13 18:00:15 2024].0765, 0.3099: GDAL:
GDALOpen(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x560e4d0afb70) succeeds as GTiff.
GDAL ERROR 0: [Fri Sep 13 18:00:15 2024].0956, 0.3291: GDAL: Computing area
of interest: 6.62976, 46.6391, 6.63534, 46.6422
[Fri Sep 13 18:00:15 2024].0958, 0.3293: GTiff: ScanDirectories()
[Fri Sep 13 18:00:15 2024].0959, 0.3294: GTiff: Opened band mask.
[Fri Sep 13 18:00:15 2024].0961, 0.3295: GTiff: Opened 8929x7207 overview.
[Fri Sep 13 18:00:15 2024].0962, 0.3296: GTiff: Opened 4464x3603 overview.
[Fri Sep 13 18:00:15 2024].0962, 0.3297: GTiff: Opened 2232x1801 overview.
[Fri Sep 13 18:00:15 2024].0963, 0.3298: GTiff: Opened 1116x900 overview.
[Fri Sep 13 18:00:15 2024].0964, 0.3299: GTiff: Opened 558x450 overview.
[Fri Sep 13 18:00:15 2024].0965, 0.3300: GTiff: Opened 279x225 overview.
[Fri Sep 13 18:00:15 2024].0966, 0.3301: GTiff: Opened band mask for
8929x7207 overview.
[Fri Sep 13 18:00:15 2024].0967, 0.3302: GTiff: Opened band mask for
4464x3603 overview.
[Fri Sep 13 18:00:15 2024].0968, 0.3303: GTiff: Opened band mask for
2232x1801 overview.
[Fri Sep 13 18:00:15 2024].0969, 0.3303: GTiff: Opened band mask for
1116x900 overview.
[Fri Sep 13 18:00:15 2024].0970, 0.3304: GTiff: Opened band mask for
558x450 overview.
[Fri Sep 13 18:00:15 2024].0971, 0.3305: GTiff: Opened band mask for
279x225 overview.
[Fri Sep 13 18:00:15 2024].2011, 0.4346: GDAL:
GDALOpen(/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff,
this=0x7f0dfc08dfd0) succeeds as GTiff.
GDAL ERROR 0: [Fri Sep 13 18:00:15 2024].2120, 0.4454: GDAL: Computing area
of interest: 6.62976, 46.6391, 6.63534, 46.6422
[Fri Sep 13 18:00:15 2024].2121, 0.4456: GTiff: ScanDirectories()
[Fri Sep 13 18:00:15 2024].2122, 0.4457: GTiff: Opened band mask.
[Fri Sep 13 18:00:15 2024].2123, 0.4457: GTiff: Opened 8929x7207 overview.
[Fri Sep 13 18:00:15 2024].2123, 0.4458: GTiff: Opened 4464x3603 overview.
[Fri Sep 13 18:00:15 2024].2124, 0.4459: GTiff: Opened 2232x1801 overview.
[Fri Sep 13 18:00:15 2024].2125, 0.4459: GTiff: Opened 1116x900 overview.
[Fri Sep 13 18:00:15 2024].2125, 0.4460: GTiff: Opened 558x450 overview.
[Fri Sep 13 18:00:15 2024].2126, 0.4460: GTiff: Opened 279x225 overview.
[Fri Sep 13 18:00:15 2024].2126, 0.4461: GTiff: Opened band mask for
8929x7207 overview.
[Fri Sep 13 18:00:15 2024].2127, 0.4462: GTiff: Opened band mask for
4464x3603 overview.
[Fri Sep 13 18:00:15 2024].2128, 0.4462: GTiff: Opened band mask for
2232x1801 overview.
[Fri Sep 13 18:00:15 2024].2128, 0.4463: GTiff: Opened band mask for
1116x900 overview.
[Fri Sep 13 18:00:15 2024].2129, 0.4463: GTiff: Opened band mask for
558x450 overview.
[Fri Sep 13 18:00:15 2024].2129, 0.4464: GTiff: Opened band mask for
279x225 overview.
[Fri Sep 13 18:00:15 2024].2130, 0.4465: S3: Downloading 16384-32767 (
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 18:00:15 2024].3454, 0.5789: S3: Got response_code=206
[Fri Sep 13 18:00:15 2024].3456, 0.5791: S3: Downloading 180224-589823 (
https://prod-pix4d-cloud-eu-central-1.s3.amazonaws.com/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff).
..
[Fri Sep 13 18:00:15 2024].4272, 0.6606: S3: Got response_code=206
[Fri Sep 13 18:00:15 2024].4276, 0.6610: GDAL: GDAL_CACHEMAX = 1591 MB

What can it be?

BTW, in both cases it says
ERROR: Status 2: File
/vsis3/prod-pix4d-cloud-eu-central-1/user-5b21721a-90d0-49c9-9d1c-d2a0c5929be5/project-1969386/ortho_display.tiff
could not be found
And later it opens it. I don't know why.

Thank you very much
Javier.

On Mon, 9 Sept 2024 at 23:49, Mayer, David P <dpmayer at usgs.gov> wrote:

> Would ~200 GB be large enough?
>
> Here is a mosaic of a section of Mars at 6 meters/pixel in a public S3
> bucket:
>
> s3://asc-pds-services/mosaic/Mars/Mars_MRO_CTX_Equi_Mosaics_Robbins/6mpp/MC16_6mpp.16bit.tif
>
> Cheers,
> David
>
> -----------
> David Mayer
> Geographer
> Astrogeology Science Center
> US Geological Survey
> dpmayer at usgs.gov
> ------------------------------
> *From:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of Javier
> Jimenez Shaw via gdal-dev <gdal-dev at lists.osgeo.org>
> *Sent:* Monday, September 9, 2024 13:55
> *To:* gdal dev <gdal-dev at lists.osgeo.org>
> *Subject:* [EXTERNAL] [gdal-dev] s3 request example
>
>
>
>
> * This email has been received from outside of DOI - Use caution before
> clicking on links, opening attachments, or responding.  *
>
>
>
> Hi
>
> I am experiencing a much slower load of a remote COG using vsis3 with the
> QGIS from conda than the QGIS in my Ubuntu (that uses GDAL 3.4.1).
>
> Because both are QGIS 3.38, I think the problem is in the version of GDAL.
> I would like to research it more.
>
> Is there any example of a (big) COG file in Amazon S3 I can use to compare
> them? Unfortunately the credentials for the files I am using expire after a
> few hours, so they are useless to report anything.
>
> Best regards,
> Javier
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240913/f3423b48/attachment-0001.htm>