[pdal] Replacing Arbiter with GDAL's VSI?

[Paul Harwood]

+1

On Thu, 23 Jan 2025, 19:16 Kirk Waters - NOAA Federal via pdal, <
pdal at lists.osgeo.org> wrote:

> Howard,
> I'm in favor of your proposal to switch to VSI. Admittedly, I haven't used
> Arbiter much, but I have used the VSI mechanisms.
>
> Kirk Waters, PhD
> NOAA Office for Coastal Management
> Applied Sciences Program
> coast.noaa.gov/digitalcoast
>
>
>
>
> On Thu, Jan 23, 2025 at 1:48 PM Howard Butler via pdal <
> pdal at lists.osgeo.org> wrote:
>
>> All,
>>
>> Recent tickets and pull requests to PDAL have shown that Arbiter's
>> capabilities, especially for cloud object storage platforms that aren't
>> AWS, are lacking in a number of areas. PDAL has previously used the Arbiter
>> library [1] from Connor Manning to provide its virtual file access, but it
>> wasn't designed to act in this role from the beginning, and it has evolved
>> piecewise to be where it is today. Do we really want to keep investing to
>> add capability there when it already exists in one of our dependencies?
>>
>> GDAL's VSI [2] has also evolved to provide a virtual access layer, but it
>> has more features, a wider testing footprint, and covers many more types of
>> remote/alternative resources. Because PDAL already has a hard GDAL
>> dependency, I'm writing to see if the user community would be enthusiastic
>> about PDAL dropping Arbiter and leveraging VSI going forward.
>>
>> A few things could help make this switchover less disruptive. First, PDAL
>> 2.9 will introduce something called a "FileSpec" [3] that will allow the
>> 'filename' in a pipeline to be an object with an arbitrary number of nodes
>> under it. These could include typical configuration options. Second, I
>> suspect that many are not fans of VSI's syntax. We would presumably allow
>> users to specify 'filename' using VSI syntax, but also provide the ability
>> for users to use proper protocol prefixes and define all security and other
>> configuration information via the FileSpec.
>>
>> What say you?
>>
>> Howard
>>
>> [1] https://github.com/connormanning/arbiter
>> [2] https://gdal.org/en/stable/user/virtual_file_systems.html
>> [3] https://github.com/PDAL/PDAL/pull/4625
>> _______________________________________________
>> pdal mailing list
>> pdal at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/pdal
>>
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20250123/7eeb2651/attachment.htm>