[gdal-dev] how to use tiledb datasets on gcs?

[Vincent Schut]

Even's understanding is correct. I've made a small PR to add this 
functionality: https://github.com/OSGeo/gdal/pull/3728. I'm not an 
experienced c++ programmer, so I hope I've done things the right way. At 
least it works here.

Let's move the rest of the discussion - if any - to the PR.

Vincent.

On 4/23/21 4:20 PM, Even Rouault wrote:
>
> Sean,
>
> my understanding of the commit is that internally the tiledb SDK 
> handles cloud storage by itself (ie not relying on GDAL virtual file 
> systems) and using s3:// / gcs:// naming conventions, and the purpose 
> of the below referenced commit is, for the GDALOpen() code path, to 
> remap /vsis3/ to s3:// , since that's what tiledb SDK expects. But 
> Norman might better comment on.
>
> Even
>
> Le 23/04/2021 à 16:14, Sean Gillies a écrit :
>> Hi Vincent, Even.
>>
>> Why would we do this when /vsigcs/... should work? Letting individual 
>> drivers set their own conventions for dataset names seems, to me, to 
>> hurt long-term maintenance and interoperability.
>>
>> On Fri, Apr 23, 2021 at 7:34 AM Vincent Schut <schut at satelligence.com 
>> <mailto:schut at satelligence.com>> wrote:
>>
>>     Thanks for confirming, Even. That doesn't look too difficult.
>>     I'll give it a try.
>>
>>     On 4/23/21 3:09 PM, Even Rouault wrote:
>>>
>>>     I guess you should add something similar to
>>>     https://github.com/OSGeo/gdal/commit/3623f9c91a2c513af204d30fe25314dbe5c7b9be
>>>     <https://github.com/OSGeo/gdal/commit/3623f9c91a2c513af204d30fe25314dbe5c7b9be>
>>>     for /vsigs
>>>
>>>     Le 23/04/2021 à 14:59, Vincent Schut a écrit :
>>>>     On 4/23/21 2:17 PM, Vincent Schut wrote:
>>>>>     Hi, how should I specify a tiledb dataset's url that resides
>>>>>     on gcs (google cloud storage) to gdal? I've tried several
>>>>>     combinations of gcs://, /vsigs/, prefixed with TILEDB:// or
>>>>>     not, but no luck. I've looked in the driver source, and
>>>>>     apparently there is only a /vsis3/ -> tiledb uri translation,
>>>>>     but no equivalent gcs one?
>>>>>
>>>>     To clarify this a bit: writing works:
>>>>
>>>>     gdal_translate -of tiledb -co COMPRESSION=ZSTD
>>>>     S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tif
>>>>     gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb
>>>>
>>>>     and the file is correctly created in the bucket. However, when
>>>>     I want to open it, it fails:
>>>>
>>>>     gdalinfo -if tiledb
>>>>     gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb
>>>>     ERROR 4:
>>>>     gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb:
>>>>     No such file or directory
>>>>     gdalinfo failed - unable to open
>>>>     'gcs://s11-dev-vincent-tiledb-test-public/S2B_MSIL1C_20210227T032659_N0209_R018_T47MQV_20210227T072433.tiledb'.
>>>>
>>>>     Is there any way to correctly specify this tiledb url on gcs
>>>>     for gdal?
>>>>
>>>>     Vincent.
>>
>> -- 
>> Sean Gillies
> -- 
> http://www.spatialys.com
> My software is free, but my time generally not.

-- 

	

Vincent Schut

Remote Sensing Software Engineer

+31 302272679 ~ Maliebaan 22 | 3581CP | Utrecht | Netherlands

Linkedin <https://www.linkedin.com/company/satelligence/>~ 
satelligence.com <http://www.satelligence.com><http://www.satelligence.com>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210423/97d6e25d/attachment.html>