[gdal-dev] Failing to write to s3 using GTiff driver.

[William Roper]

Thanks for helping track it down!

On Tue, 16 Aug 2022 at 14:05, Even Rouault <even.rouault at spatialys.com>
wrote:

> William,
>
> COG: Generating overviews of the imagery: start
>> S3: GetFileSize(
>> https://my-bucket.s3.amazonaws.com/test_batch_output/test.tif.ovr.tmp)=0
>>  response_code=404
>> S3: /vsis3/my-bucket/test_batch_output/test.tif.ovr.tmp is not a object
>> GTiff: File being created as a BigTIFF.
>> ERROR 1: Only read-only mode is supported for /vsicurl
>> COG: Generating overviews of the imagery: end
>> S3: UploadId:
>> Rok8InTcYwFMoHYA3kVAwlLDycIxreczdLSODBpLA5exAshfS46ZclK.3.2.a_EAAY3vhoJQnc7DsVk9X3pzsRetSAZ.oc39kDwRe755ftroj5rH9BRXLPaNCcnuTGZP
>> S3: Etag for part 1 is "52647419376831771b40964bad80d86e"
>> S3: Etag for part 2 is "4bea862d0878bbe1dbe717aa8cea48c8"
>> S3: Etag for part 3 is "5ba3c9e4ea924f06cdc2ee614e5627c0"
>> S3: Etag for part 4 is "07e53a2e07a2575d1b2d89cf8040ebe8"
>> GDAL:
>> GDALClose(/vsis3/my-bucket/test_batch_output/test.tif.warped.tif.tmp,
>> this=0x55a288913060)
>> S3: GetFileSize(
>> https://my-bucket.s3.amazonaws.com/test_batch_output/test.tif.warped.tif.tmp)=207439264
>>  response_code=206
>> GDAL: GDALClose(/vsis3/my-bucket/dtm/test.tif, this=0x55a2882e8570)
>> GDAL: In GDALDestroy - unloading GDAL shared library.
>>
>
>  It looks to me like gdal attempted to write a lot of the tmp files to s3,
> rather than locally? the command I used is:
>
> Yes exactly this is an issue with the temporary file created when
> computing overviews, before integrating them to the final COG file. I've
> tracked this issue at https://github.com/OSGeo/gdal/issues/6205 . I
> should be able to look at fixing that soon
>
> Even
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220816/04839b04/attachment.htm>