[gdal-dev] Call for discussion on RFC 59 (v2): GDAL/OGR utilities as a library

[Even Rouault]

Folks,

I'd wish we can find a solution that satisfy all parties. Here's another iteration for a possible proposition.

* C API:

Principles :
- dataset objects can be passed
- the option structure is opaque (should address Frank's concern about exposing too much internal stuff)
- the option structure is parsed from an array of strings (with the current syntax of utilities)
- a few setters can be added, for example, to set a progress function. Or possibly auxiliary objects,
  like the cutline layer for gdalwarp.
- not sure if we need the *pbUsageError flag. I removed it. If there's a conflict of options only found after
  parsing time (eg due to characteristics of the datasets), then a CPLError() message should be enough.

Example with GDALTranslate:

char** papszArgv = CSLParseCommandLine(const char* pszCommandLine);
GDALTranslateOptionsParse* psOptions = GDALTranslateOptionsNew(char** papszArgv);
GDALTranslateOptionsSetProgress(options, pfnProgress, pProgressData);
hOutDS = GDALTranslate(const char *pszDest, GDALDatasetH hSrcDataset, GDALTranslateOptions *psOptions)
GDALTranslateOptionsDestroy(GDALTranslateOptionsParse* psOptions);
CSLDestroy(papszArgV);


* Python API

Principles :
- use the above C API from SWIG, but mostly for internal use of the upper levels explained below that
  would be only Python code (no SWIG).
- pubic API offers access to (hopefully nicely named) dedicated arguments and builds the string from them
- public API offers access to the string representation as well
- use Python dynamic typing to offer sugar candy, e.g pass a SRS either as an osr.SpatialReference()
object or a string (the object being serialized to string, but this is a lossless operation)


Examples:

1) Use case with repeatable options
options = gdal.TranslateOptions()
options.bands = [ 1, 2, 3 ]
options.format = 'MEM'
options.progress = my_progress_method
mem_ds = gdal.Translate('', src_ds, options = options)

2)  Variant of 1). With some Python magic on **kwargs it can be automated to redirect on 1)
mem_ds = gdal.Translate('', src_ds, bands = [1,2,3], format = 'MEM', progress = my_progress_method)

3) String oriented.
options = gdal.TranslateOptions('-b 1 -b 2 -b 3 -of MEM')
mem_ds = gdal.Translate('', src_ds, options = options, progress = my_progress_method)

4) Variant of 4)
mem_ds = gdal.Translate('', src_ds, options = '-b 1 -b 2 -b 3 -of MEM', progress = my_progress_method)

5) For the nostalgics, a wrapper of the above :
gdal.Translate('in.tif out.tif -b 1 -b 2 -b 3 -of MEM', progress = my_progress_method')

That's maybe too many different possibilities, although some build upon others, so not necessarily
a lot of code involved (easier to say when it is not coded ahah!)

* Other binding languages.

Would probably only fallback on wrapping C API with SWIG in a first step.
And find most appropriate solutions for each language.

- Java for sure doesn't have a keyword approach for method arguments
(well with what I remember from Java 1.6. might have changed). Builder approach
would be a possibility (http://stackoverflow.com/questions/1988016/named-parameter-idiom-in-java) 
- C#: being a clone of Java, probably not.
- Perl: apparently possible


Opinions ?

Even


> Hi Frank,
> 
> I was one of the original people who argued against the "array of strings"
> approach...
> 
> On 27 August 2015 at 02:26, Frank Warmerdam <warmerdam at pobox.com> wrote:
> > I clearly should have been commenting sooner.
> 
> Several months ago :p
> 
> > I am concerned that having messy structures of options for each
> > program is going to complicate maintaining the actually commandline
> > programs, and that it will still be a fragile and complicated point of
> > entry as commandline arguments evolve over time.
> 
> The commandline tools eventually become string-parsing and wrapping of the
> corresponding library tool - I'm not sure that makes it fragile &
> complicated? Means that the library-ified apps are *at least as*
> flexible/expressive/powerful
> as the command-line tools.
> 
> > I'd prefer if the approach had just been to embed the main()'s in a
> > library and to still pass the exact same vector of arguments (in the
> > char **argv format) to these functions instead of shelling out a
> > program.
> 
> That kinda defeats the whole point - a huge array of complex string-ified
> arguments is what we're all doing at the moment, wrapped in an subprocess
> call. Some options take multiple arguments in multiple strings, others take
> multiple arguments in single strings, it's massively confusing. And we all
> have big pipelines of chained gdalwarp/gdal_translate/etc code...
> 
> What we were striving for was to make it distinctly *better*:
> 
>    - progress/logging/error handling
>    - options that accept geometries or SRS or in-memory datasets without
>    having to re-serialize them and/or utilise tempfiles
>    - easily applying the same operations over multiple datasets
>    - configuration option defaults
> 
> > I would love to be able to replace many places where I shell out to
> > run gdal command line programs with a library call with essentially
> > the same arguments.
> 
> Sure, so it should be straightforward to do that *as well*, though besides
> in-memory data (as you mention) you're getting very little benefit.
> 
> Rob :)

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com