[gdal-dev] zarr driver PR

[Ivan Lucena]

Hi Sean,

I started playing with the idea of building a Zarr driver a year ago. I haven't done much at that time and haven't touched it since.

Even though I have developed a bunch of drivers in C++, with various forms of compression,  since the beginning it occurred to me that the Zarr driver could be better off if developed in Python.

Just an idea.

Good luck,

Ivan

________________________________
From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of David Brochart <david.brochart at gmail.com>
Sent: Monday, February 1, 2021 11:14 AM
To: Sean Gillies <sean.gillies at gmail.com>
Cc: Gdal Dev <gdal-dev at lists.osgeo.org>
Subject: Re: [gdal-dev] zarr driver PR

Hi Sean,

Thanks for starting this discussion. Let me first say that the PR is still in an early stage, and that more work is needed upstream to make it fully usable, so definitely not ready for merging yet.
Although it is not clearly mentioned, xtensor-zarr supports Zarr v3 as well as a part of Zarr v2, actually the test in the PR is based on version 2. But as you mentioned, supporting all the features that are allowed with zarr-python is not going to be possible in C++. We will most likely limit the range of possibilities, and improve the driver based on interest in some features. For instance, this PR starts very modestly by not supporting any geo-metadata.
In any case, this PR is nothing more than a work in progress at the moment, without any formal scope. I'm interested in getting feedback from the GDAL community, and if there is interest we can work on formalizing this work through an RFC, as it seems it is going to be required for new drivers.

On Mon, Feb 1, 2021 at 4:46 PM Sean Gillies <sean.gillies at gmail.com<mailto:sean.gillies at gmail.com>> wrote:
Hi all,

Also deep in our recent long thread about removing drivers from GDAL is mention of a new PR for a "zarr" driver.

https://github.com/OSGeo/gdal/pull/3411

I'd like to see more discussion of the scope of this work and plans for future development and support before a zarr driver is added to GDAL.

Is the zarr driver not going to support version 2 of the zarr spec? I that v2 would be difficult to implement as a GDAL driver as it is extremely flexible and extensible. For example, I've written custom compressor and storage for use in a project at work and can't imagine how a driver written in C++ would understand it.

Is the zarr driver based on https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html? That work seems not yet finished. Is it premature to add a driver to GDAL for a format that isn't yet specified and may yet be changed? Or should we expect only minor changes at this point?

--
Sean Gillies
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210201/79402b1f/attachment.html>