[GRASS-git] [OSGeo/grass] bca6f4: grass.tools: Add API to access tools as functions ...

[Vaclav Petras]

  Branch: refs/heads/main
  Home:   https://github.com/OSGeo/grass
  Commit: bca6f4334c2ad919433fba9fd4e93ae44cd60160
      https://github.com/OSGeo/grass/commit/bca6f4334c2ad919433fba9fd4e93ae44cd60160
  Author: Vaclav Petras <wenzeslaus at gmail.com>
  Date:   2025-07-27 (Sun, 27 Jul 2025)

  Changed paths:
    M python/grass/CMakeLists.txt
    M python/grass/Makefile
    M python/grass/app/cli.py
    M python/grass/app/tests/grass_app_cli_test.py
    M python/grass/experimental/Makefile
    M python/grass/experimental/tests/conftest.py
    A python/grass/tools/Makefile
    A python/grass/tools/__init__.py
    A python/grass/tools/session_tools.py
    A python/grass/tools/support.py
    A python/grass/tools/tests/conftest.py
    A python/grass/tools/tests/grass_tools_result_test.py
    A python/grass/tools/tests/grass_tools_session_tools_test.py
    A python/grass/tools/tests/grass_tools_tool_function_wrap_test.py

  Log Message:
  -----------
  grass.tools: Add API to access tools as functions (#2923)

This adds a Tools class which allows to access GRASS tools (modules) to be accessed using methods. Once an instance is created, calling a tool is calling a function (method) similarly to grass.jupyter.Map. Unlike grass.script, this does not require a general function to be called with a tool name as a parameter, and unlike grass.pygrass module shortcuts, this does not require special objects to mimic the module families just to get tool names into Python syntax.

Outputs are handled through a returned object which is result of automatic capture of outputs and can do conversions from known formats using properties. The capture is opt-out rather than opt-in as with the other interfaces. The result object is, in a way, similar to what subprocess.run function returns, but focused on GRASS text formats.

The code is included under a new grass.tools package and is marked as experimental which allows merging the code while doing breaking changes after a release.

The implementation maximizes sharing of the low-level code with grass.script, so handle_errors, Popen, and parameter processing is reused, but with different defaults (exceptions and output capturing). It relies on grass.script.Popen which newly defaults to text=True. Both stdout and stderr are passed as is to the result (so as string by default and bytes when text=False).

Adds also a run subcommand to have a CLI use case for the tools. It runs one tool in XY project, so useful only for things like g.extension or m.proj, but, with a workaround for argparse --help, it can do --help for a tool. Processing of special flags is more robust in grass.tools to accommodate the CLI usage.

Uses the no-copy approach for env taken also elsewhere like init which allows for the changes in the session to be reflected in the tools. This requires overwrite and verbosity setting to be handled in an internal, ad-hoc copy which is avoided if not needed. No special computational region treatment.

Dynamic module attributes are used instead of imports, anticipating standalone tools to live in the same package and users picking one or the other but not both. (This requires silencing a Pylint, but we also could not use the all attribute at all.) The use of lazy imports enables code from grass.tools import Tools without forcing that import on everyone importing anything from the other modules (like standalone tools). The file layout itself should accommodate future additions such as standalone tools.

Using io.StringIO for stdin assumes that any tool which will have stdin will have something like input=- or file=- because there is no other way of getting stdin data to the tool at this point. The cmd functions taking a command as a list take an input parameter for stdin aligning with subprocess.run and Popen.communicate (the name stdin is already used for a pipe flag parameter). Allows for stderr to be captured even without capturing stdout to allow for exceptions with error message while simply printing stdout.

The Tools class can also behave like a context manager which is useful when used with other context managers and also will be useful when more functionality is added (like pack file IO) or for aligning (feature parity) with other Tools-like implementations which will require resource handling (like the standalone tools).

The four different tool calling functions (when tool names are not used as attributes) use short names to always keep the focus on the tool, distinguish smart and basic behavior by run and call, and parameter-based (kwargs) and list (command) by no suffix and _cmd suffix.

Function-like calling of tools:

* Tool failure causes an exception with an error message being part of the exception (traceback).
* All returncode, standard output, and error output are part of one result object similar to what subprocess.run returns.
* Access and post-processing of the text result (standard output) is done by the result object (its properties or methods).
* Additionally, the result object can be directly indexed to access a JSON parsed as a Python list or dictionary.
* Standard input is passed as `parameter_name=io.StringIO` which takes care of `input="-"` and piping the text into the subprocess.
* A _session_ or _env_ is accepted when creating an object (`Tools(session=session)`). No need to pass _env_ later.
* `__dir__` code completion for function names and `help()` calls work and work even outside of a session.
* The keyval format processing converts strings to ints and floats.

Other functionality:

* High-level and low-level usage is possible with four different functions: taking Python function parameters as tool parameters or taking a list of strings forming a command, and processing the parameters before the subprocess call or leaving the parameters as is (2x2 matrix).
* Handling of inputs and outputs can be customized (e.g., not capturing stdout and stderr).
* The _env_ parameter is also accepted by individual functions (as with _run_command_).
* The Tools object is a context manager, so it can be created with other context managers within one `with` statement and can include cleanup code in the future.
* Tools-object-level overwrite and verbosity setting including limited support for the reversed (False) setting which is not possible with the standard flags at the tool level (there is no `--overwrite=False` in CLI).

Features originally implemented, but at the end removed:

* Region freezing when Tools object is created.
* Allow to specify stdin and use a new instance of Tools itself to execute with that stdin through a new function which returns that instance (feed_input_to method and stdin in the constructor).
* A special function to ignore errors returning a new instance.
* Overwriting by default like in grass.jupyter.
* Wrappers with names from the run_command family.
* Processing of command line call into JSON which is useful only for the pack file IO.
* Tools in grass.experimental as opposed to grass.tools only marked as experimental.



To unsubscribe from these emails, change your notification settings at https://github.com/OSGeo/grass/settings/notifications