[gdal-dev] Call for discussion on RFC 63 : Sparse datasets improvements

[Kurt Schwehr]

Scratch my enum comment...

http://stackoverflow.com/questions/1448396/how-to-use-enums-as-flags-in-c

On Mon, Jul 11, 2016 at 8:31 AM, Even Rouault <even.rouault at spatialys.com>
wrote:

> Le lundi 11 juillet 2016 17:11:59, Kurt Schwehr a écrit :
> > Even,
> >
> > I think this looks good and a big win for a lot of use cases.
> >
> > Comments related to the flags defined in gcore/gdal.h:
> >
> > - If possible, use an enum rather than bare defines.
>
> They are flags that can be OR'ed, so you cannot really define an
> enumeration. Or
> am I missing something ?
>
>
> > - Add comments to gdal.h explaining what the values are
> Yep, will do
>
> > - Should an unimplemented also have the _DATA flag set rather than
> _EMPTY?
>
> Yes, this is the case of the default
> GDALRasterBand::IGetDataCoverageStatus().
> I should also mention it in the doc that a driver that would implement the
> method, but in some cases would want to return _UNIMPLEMENTED should also
> set
> _DATA.
>
> >
> > A suggestion for this
> > <
> https://github.com/OSGeo/gdal/compare/trunk...rouault:sparse_datasets?expa
> > nd=1#diff-66f7e256611c32cddc2745d78b4fe59bR3787>, which looks brittle to
> > me:
> >
> >      if( nStatus != GDAL_DATA_COVERAGE_STATUS_EMPTY )
> >
> > A helper function to check empty rather than a direct check will allow
> for
> > later or'ing the status with other flags.  Or you can & with the flag.
> But
> > looking at it a second time, I think it might be more explicit if you
> did a
> > check against having data:
> >
> >     if( nStatus & GDAL_DATA_COVERAGE_STATUS_DATA )
>
> Yes, that's probably better (assuming that _unimplemented comes with _data
> set)
>
> Thanks
>
> >
> > -kurt
> >
> > On Sun, Jul 10, 2016 at 2:54 AM, Even Rouault <
> even.rouault at spatialys.com>
> >
> > wrote:
> > > Le dimanche 10 juillet 2016 11:32:48, Andrew C Aitchison a écrit :
> > > > On Fri, 8 Jul 2016, Even Rouault wrote:
> > > > > The topic of sparse dataset management come back regularly, so I've
> > > > > decided to tackle it.
> > > > >
> > > > > Please find
> > > > >
> https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements
> > >
> > > for
> > >
> > > > > review.
> > > >
> > > > I know of several proprietary file formats where the data is tiled
> > > > with the index indicating (perhaps implicitly) where the tile has no
> > > > data.
> > > >
> > > > A driver for such formats could have IReadBlock quickly return with
> > > > a code to indicate NoData, rather than filling in the image data.
> > > > As it stands that might mean extending CPLErr, but would
> > > > that be helpful to the main library ?
> > >
> > > That would have a significant impact on the whole code base, as well as
> > > application code, so I didn't really considered that option and
> prefered
> > > an auxiliary interface to be used by code aware and caring about
> special
> > > behaviour for sparse datasets.
> > >
> > > > Is this what is described by having the offset and byte count both
> zero
> > > > ?
> > >
> > > Yes, in TIFF, you have 2 arrays, one that indicates at which offset in
> > > the file
> > > a given tile/strip is located, and the other one for the number of
> bytes
> > > of that tile/strip. GDAL uses offset = count = 0 as a convention for
> > > missing blocks.
> > >
> > > > ----
> > > >
> > > > I don't really understand how GDAL_DATA_COVERAGE_STATUS values
> combine
> > > > or
> > >
> > > * If the requested window contains has no missing blocks, it returns
> > > GDAL_DATA_COVERAGE_STATUS_DATA
> > > * If the requested window has only missing blocks, it returns
> > > GDAL_DATA_COVERAGE_STATUS_EMPTY
> > > * If the requested window is a mix of both, it returns
> > > GDAL_DATA_COVERAGE_STATUS_DATA | GDAL_DATA_COVERAGE_STATUS_EMPTY
> > >
> > > > when pdfDataPct is valid.
> > >
> > > It should be valid if the processing has not been stopped prematurely
> due
> > > to
> > > the nMaskFlagStop being triggered. For example if you have a dataset
> and
> > > you
> > > want a special processing (could be just an info "This dataset is
> > > sparse") as
> > > soon as it contains empty blocks, then you can query the whole dataset
> > > extent
> > > with nMaskFlagStop = GDAL_DATA_COVERAGE_STATUS_EMPTY. As soon as a
> > > missing block is found, the function will exit, and will thus be unable
> > > to determine
> > > the percentage of valid data.
> > >
> > > > In one of the formats above, the tile index has special values
> > > > for "no data" and and for "data exists and could be
> retrieved/purchased
> > > > if required". I'd consider mapping these to
> > >
> > > GDAL_DATA_COVERAGE_STATUS_EMPTY
> > >
> > > Clearly a missing block will cause GDAL_DATA_COVERAGE_STATUS_EMPTY to
> be
> > > set.
> > >
> > > > and GDAL_DATA_COVERAGE_STATUS_UNIMPLEMENTED. Does that make sense ?
> > >
> > > GDAL_DATA_COVERAGE_STATUS_UNIMPLEMENTED is aimed at being returned
> when a
> > > driver does not offer an implementation of the interface, and thus uses
> > > the default dumb implementation that returns
> > > GDAL_DATA_COVERAGE_STATUS_UNIMPLEMENTED
> > >
> > > I realize that I didn't really document yet the semantics of those
> flags.
> > > To be
> > > done.
> > >
> > >
> > > --
> > > Spatialys - Geospatial professional services
> > > http://www.spatialys.com
> > > _______________________________________________
> > > gdal-dev mailing list
> > > gdal-dev at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>



-- 
--
http://schwehr.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20160711/1cbcb795/attachment-0001.html>