[gdal-dev] Problem when trying to open files with GDAL from Object Store using vsiswift

[ZAZHIL-HA HERENA]

Hello everyone,

I am using GDAL 2.4.1  with Python 3.6 to access rasters in Object Storage using /vsiswift. I need to open around 100 rasters, read their data and capture it in an array. I do not need to have them all available in memory, this is a sequential task , read one raster, delete dataset and continue with another one within a for loop.

Here is a fragment of my code:

 files = ['/vsiswift/spatial/sourceinput/Atlanta_nadir27_catid_1030010003472200/Pan-Sharpen/Pan-Sharpen_Atlanta_nadir27_catid_1030010003472200_732701_3725139.tif',
             '/vsiswift/spatial/sourceinput/Atlanta_nadir42_catid_10300100035D1B00/Pan-Sharpen/Pan-Sharpen_Atlanta_nadir42_catid_10300100035D1B00_732701_3725139.tif',
             '/vsiswift/spatial/sourceinput/Atlanta_nadir7_catid_1030010003D22F00/Pan-Sharpen/Pan-Sharpen_Atlanta_nadir7_catid_1030010003D22F00_732701_3725139.tif',
              .
              .
              .
             '/vsiswift/spatial/sourceinput/Atlanta_nadir42_catid_10300100035D1B00/Pan-Sharpen/Pan-Sharpen_Atlanta_nadir42_catid_10300100035D1B00_732701_3730989.tif'
             ]
 for _file in tqdm(files[:]):
    tileHdl = gdal.Open(_file, gdal.GA_ReadOnly)
    data = tileHdl.ReadAsArray()
    del tileHDL


My problem is that, after reading 19 rasters, the 20th one returns a dataset in the Open operation , but returns 'None' when I try the 'ReadAsArray()'. The 20th raster is a valid one, if I move this "failing" raster to the first position in the loop it gets read successfully.

I did print gdal lastErrorMsg and got the following error: "band 4: IReadBlock failed at X offset 0, Y offset 0: TIFFReadEncodedStrip() failed"

Has anyone else succeded in accessing multiple rasters in the same python program? Is this a known issue? Maybe I have a problem in my code?

Truly appreciate your help and time.

Thanks,

Zazhil-ha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20190410/8230d5ae/attachment.html>