[gdal-dev] Errors while creating sozip zarr file

[Erik Schnetter]

I am using GDAL to create a multidimensional zarr file that is sozip compressed. I see this error when creating the file:

ERROR 1: dish_positions.00000000.zarr/zarr.json already exists in ZIP file
ERROR 8: Open file /vsizip/data/fengine_init_pathfinder/cx66_dish_positions.00000000.zarr.zip/dish_positions.00000000.zarr/zarr.json to write failed

Everything is working fine when I do not use sozip compression. I enable sozip compression by adding a "/vsizip" prefix to the file name. Although there is an error reported on screen, I do not see an error code reported by the function creating or closing the multidimensional dataset. The resulting file ("*.zarr.zip") is created fine and looks almost correct, but all attributes seem to be missing.

I wonder – is it actually possible to create a zarr file that is sozip compressed, given that zarr probably writes to each of its file multiple times? If not, what is the preferred way to create a sozip-compressed zarr file efficiently?

Some details:

I create the dataset (i.e. the file) via

                const auto driver_manager = GetGDALDriverManager();
                const auto driver = driver_manager->GetDriverByName("Zarr");
                const auto dataset = std::unique_ptr<GDALDataset>(driver->CreateMultiDimensional(
                    full_path.c_str(), root_group_options_c.data(), options_c.data()));

where "full_path" is "/vsizip/data/fengine_init_pathfinder/cx66_dish_positions.00000000.zarr.zip/dish_positions.00000000.zarr".

I then create multiple attributes ("CreateAttribute") and then

                const auto mdarray = group->CreateMDArray(meta->get_name(), dimensions, datatype,
                                                          array_options_c.data());
                    const bool success = mdarray->Write(
                        arrayStart.data(), count.data(), nullptr, bufferStride.data(), datatype,
                        frame + datatypesize * meta->offset, frame, buffer->frame_size);

and finish with

                const CPLErr err = dataset->Close();
                assert(!err);

The full code is available at <https://github.com/kotekan/kotekan/blob/eschnett/updates-2/lib/stages/gdalFileWrite.cpp>.

-erik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250719/ef60f4de/attachment-0001.htm>