[gdal-dev] how to use tiledb datasets on gcs?

[Norman Barker]

Yes, internally the GDAL TileDB driver remaps vsis3 requests to s3 requests
within the TileDB library. To a user the vsi notation still works the same.

I will look over the PR, thanks Vincent!

Norman


On Fri, Apr 23, 2021 at 9:30 AM Vincent Schut <schut at satelligence.com>
wrote:

> Even's understanding is correct. I've made a small PR to add this
> functionality: https://github.com/OSGeo/gdal/pull/3728. I'm not an
> experienced c++ programmer, so I hope I've done things the right way. At
> least it works here.
>
> Let's move the rest of the discussion - if any - to the PR.
>
> Vincent.
>
> On 4/23/21 4:20 PM, Even Rouault wrote:
>
> Sean,
>
> my understanding of the commit is that internally the tiledb SDK handles
> cloud storage by itself (ie not relying on GDAL virtual file systems) and
> using s3:// / gcs:// naming conventions, and the purpose of the below
> referenced commit is, for the GDALOpen() code path, to remap /vsis3/ to
> s3:// , since that's what tiledb SDK expects. But Norman might better
> comment on.
>
> Even
> Le 23/04/2021 à 16:14, Sean Gillies a écrit :
>
> Hi Vincent, Even.
>
> Why would we do this when /vsigcs/... should work? Letting individual
> drivers set their own conventions for dataset names seems, to me, to hurt
> long-term maintenance and interoperability.
>
> On Fri, Apr 23, 2021 at 7:34 AM Vincent Schut <schut at satelligence.com>
> wrote:
>
>> Thanks for confirming, Even. That doesn't look too difficult. I'll give
>> it a try.
>>
>> On 4/23/21 3:09 PM, Even Rouault wrote:
>>
>> I guess you should add something similar to
>> https://github.com/OSGeo/gdal/commit/3623f9c91a2c513af204d30fe25314dbe5c7b9be
>> for /vsigs
>> Le 23/04/2021 à 14:59, Vincent Schut a écrit :
>>
>> On 4/23/21 2:17 PM, Vincent Schut wrote:
>>
>> Hi, how should I specify a tiledb dataset's url that resides on gcs
>> (google cloud storage) to gdal? I've tried several combinations of gcs://,
>> /vsigs/, prefixed with TILEDB:// or not, but no luck. I've looked in the
>> driver source, and apparently there is only a /vsis3/ -> tiledb uri
>> translation, but no equivalent gcs one?
>>
>> To clarify this a bit: writing works:
>>
>> gdal_translate -of tiledb -co COMPRESSION=ZSTD
>> S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tif
>> gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb
>>
>> and the file is correctly created in the bucket. However, when I want to
>> open it, it fails:
>>
>> gdalinfo -if tiledb
>> gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb
>> ERROR 4:
>> gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb:
>> No such file or directory
>> gdalinfo failed - unable to open
>> 'gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb'.
>>
>> Is there any way to correctly specify this tiledb url on gcs for gdal?
>>
>> Vincent.
>>
>> --
> Sean Gillies
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
>
> --
>
> Vincent Schut
>
> Remote Sensing Software Engineer
>
> +31 302272679 ~ Maliebaan 22 | 3581CP | Utrecht | Netherlands
> Linkedin <https://www.linkedin.com/company/satelligence/> ~
> satelligence.com <http://www.satelligence.com>
> <http://www.satelligence.com>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210423/f56c5ed1/attachment.html>