[gdal-dev] Call for discussion on RFC 59 (v2): GDAL/OGR utilities as a library

[Tim Keitt]

http://www.keittlab.org/

On Tue, Sep 1, 2015 at 8:34 AM, Even Rouault <even.rouault at spatialys.com>
wrote:

> Folks,
>
> I'd wish we can find a solution that satisfy all parties. Here's another
> iteration for a possible proposition.
>
> * C API:
>
> Principles :
> - dataset objects can be passed
> - the option structure is opaque (should address Frank's concern about
> exposing too much internal stuff)
> - the option structure is parsed from an array of strings (with the
> current syntax of utilities)
> - a few setters can be added, for example, to set a progress function. Or
> possibly auxiliary objects,
>   like the cutline layer for gdalwarp.
> - not sure if we need the *pbUsageError flag. I removed it. If there's a
> conflict of options only found after
>   parsing time (eg due to characteristics of the datasets), then a
> CPLError() message should be enough.
>
> Example with GDALTranslate:
>
> char** papszArgv = CSLParseCommandLine(const char* pszCommandLine);
> GDALTranslateOptionsParse* psOptions = GDALTranslateOptionsNew(char**
> papszArgv);
> GDALTranslateOptionsSetProgress(options, pfnProgress, pProgressData);
> hOutDS = GDALTranslate(const char *pszDest, GDALDatasetH hSrcDataset,
> GDALTranslateOptions *psOptions)
> GDALTranslateOptionsDestroy(GDALTranslateOptionsParse* psOptions);
> CSLDestroy(papszArgV);
>
>
> * Python API
>
> Principles :
> - use the above C API from SWIG, but mostly for internal use of the upper
> levels explained below that
>   would be only Python code (no SWIG).
> - pubic API offers access to (hopefully nicely named) dedicated arguments
> and builds the string from them
> - public API offers access to the string representation as well
> - use Python dynamic typing to offer sugar candy, e.g pass a SRS either as
> an osr.SpatialReference()
> object or a string (the object being serialized to string, but this is a
> lossless operation)
>
>
> Examples:
>
> 1) Use case with repeatable options
> options = gdal.TranslateOptions()
> options.bands = [ 1, 2, 3 ]
> options.format = 'MEM'
> options.progress = my_progress_method
> mem_ds = gdal.Translate('', src_ds, options = options)
>
> 2)  Variant of 1). With some Python magic on **kwargs it can be automated
> to redirect on 1)
> mem_ds = gdal.Translate('', src_ds, bands = [1,2,3], format = 'MEM',
> progress = my_progress_method)
>
> 3) String oriented.
> options = gdal.TranslateOptions('-b 1 -b 2 -b 3 -of MEM')
> mem_ds = gdal.Translate('', src_ds, options = options, progress =
> my_progress_method)
>
> 4) Variant of 4)
> mem_ds = gdal.Translate('', src_ds, options = '-b 1 -b 2 -b 3 -of MEM',
> progress = my_progress_method)
>
> 5) For the nostalgics, a wrapper of the above :
> gdal.Translate('in.tif out.tif -b 1 -b 2 -b 3 -of MEM', progress =
> my_progress_method')
>
> That's maybe too many different possibilities, although some build upon
> others, so not necessarily
> a lot of code involved (easier to say when it is not coded ahah!)
>
> * Other binding languages.
>
> Would probably only fallback on wrapping C API with SWIG in a first step.
> And find most appropriate solutions for each language.
>
> - Java for sure doesn't have a keyword approach for method arguments
> (well with what I remember from Java 1.6. might have changed). Builder
> approach
> would be a possibility (
> http://stackoverflow.com/questions/1988016/named-parameter-idiom-in-java)
> - C#: being a clone of Java, probably not.
> - Perl: apparently possible
>
>
> Opinions ?
>

In an ideal world, I would prefer a nice clean algorithms library that is
orthogonal to the command line and parsing. The utilities then simply
consist of parsing and calling this library. I would also prefer the
library to be broken down in to a set of orthogonal lower-level primitives
and the higher-level algorithms built from these. But I cannot contribute
significantly to that effort so I think its up to those than can to decide
the course.

THK


>
> Even
>
>
> > Hi Frank,
> >
> > I was one of the original people who argued against the "array of
> strings"
> > approach...
> >
> > On 27 August 2015 at 02:26, Frank Warmerdam <warmerdam at pobox.com> wrote:
> > > I clearly should have been commenting sooner.
> >
> > Several months ago :p
> >
> > > I am concerned that having messy structures of options for each
> > > program is going to complicate maintaining the actually commandline
> > > programs, and that it will still be a fragile and complicated point of
> > > entry as commandline arguments evolve over time.
> >
> > The commandline tools eventually become string-parsing and wrapping of
> the
> > corresponding library tool - I'm not sure that makes it fragile &
> > complicated? Means that the library-ified apps are *at least as*
> > flexible/expressive/powerful
> > as the command-line tools.
> >
> > > I'd prefer if the approach had just been to embed the main()'s in a
> > > library and to still pass the exact same vector of arguments (in the
> > > char **argv format) to these functions instead of shelling out a
> > > program.
> >
> > That kinda defeats the whole point - a huge array of complex string-ified
> > arguments is what we're all doing at the moment, wrapped in an subprocess
> > call. Some options take multiple arguments in multiple strings, others
> take
> > multiple arguments in single strings, it's massively confusing. And we
> all
> > have big pipelines of chained gdalwarp/gdal_translate/etc code...
> >
> > What we were striving for was to make it distinctly *better*:
> >
> >    - progress/logging/error handling
> >    - options that accept geometries or SRS or in-memory datasets without
> >    having to re-serialize them and/or utilise tempfiles
> >    - easily applying the same operations over multiple datasets
> >    - configuration option defaults
> >
> > > I would love to be able to replace many places where I shell out to
> > > run gdal command line programs with a library call with essentially
> > > the same arguments.
> >
> > Sure, so it should be straightforward to do that *as well*, though
> besides
> > in-memory data (as you mention) you're getting very little benefit.
> >
> > Rob :)
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20150901/46a3f292/attachment.html>