[GRASS-git] [OSGeo/grass] 8ca3c7: grass.tools: Add raster pack files IO to Tools (#5...

[Vaclav Petras]

  Branch: refs/heads/main
  Home:   https://github.com/OSGeo/grass
  Commit: 8ca3c7a6f3a8a018c084bb575b3cebbea69f5abc
      https://github.com/OSGeo/grass/commit/8ca3c7a6f3a8a018c084bb575b3cebbea69f5abc
  Author: Vaclav Petras <wenzeslaus at gmail.com>
  Date:   2025-10-07 (Tue, 07 Oct 2025)

  Changed paths:
    A python/grass/app/tests/grass_app_cli_run_pack_test.py
    M python/grass/tools/Makefile
    A python/grass/tools/importexport.py
    M python/grass/tools/session_tools.py
    M python/grass/tools/support.py
    M python/grass/tools/tests/conftest.py
    A python/grass/tools/tests/grass_tools_session_tools_pack_test.py
    M python/grass/tools/tests/grass_tools_session_tools_test.py

  Log Message:
  -----------
  grass.tools: Add raster pack files IO to Tools (#5877)

This is adding r.pack files (aka native GRASS raster files) as input and output to tools when called through the Tools object. Tool calls such as r_grow can take r.pack files as input or output. The format is distinguished by the file extension.

Notably, tool calls such as r_mapcalc don't pass input or output data as separate parameters (expressions or base names), so they can be used like that only when a wrapper exists (r_mapcalc_simple) or, in the future, when more information is included in the interface or passed between the tool and the Tools class Python code. Similarly, tools with multiple inputs or outputs in a single parameter are currently not supported.  

The code is using --json with the tool to get the information on what is input and what is output, because all are files which may or may not exists (this is different from NumPy arrays where the user-provided parameters clearly say what is input (object) and what is output (class)). Consequently, the whole import-export machinery is only started when there are files in the parameters as identified by the parameter converter class.

Currently, the in-project raster names are driven by the file names. This will break for parallel usage and will not work for vector as is. While it is good for guessing the right (and nice) name, e.g., for r.mapcalc expression, ultimately, unique names retrieved with an API function are likely the way to go.

When cashing is enabled (either through use go context manager or explicitly), import of inputs is skipped when they were already imported or when they are known outputs. Without cache, data is deleted after every tool (function) call. Cashing is keeping the in-project data in the project (as opposed to a hidden cache or deleting them). The parameter to explicitly drive this is called use_cache (originally keep_data).

The objects track what is imported and also track import and cleaning tasks at function call versus object level. The data is cleaned even in case of exceptions. The interface was clarified by creating a private/protected version of run_cmd which has the internal-only parameters. This function uses a single try-finally block to trigger the cleaning in case of exceptions. 

While generally the code supports paths as both strings and Path objects, the actual decisions about import are made from the list of strings form of the command.

>From caller perspective, overwrite is supported in the same way as for in-project GRASS rasters.

The tests use module scope to reduce fixture setup by couple seconds. Changes include a minor cleanup of comments in tests related to testing result without format=json and with, e.g., --json option.

The class documentation discusses overhead and parallelization because the calls are more costly and there is a significant state of the object now with the cache and the rasters created in the background. This includes discussion of the NumPy arrays, too, and slightly improves the wording in part discussing arrays.

This is building on top of #2923 (Tools API, and it is parallel with #5878 (NumPy array IO), although it runs at a different stage than NumPy array conversions and uses cache for the imported data (may be connected more with the arrays in the future). This can be used efficiently in Python with Tools (caching, assuming project) and in a limited way also with the experimental run subcommand in CLI (no caching, still needs an explicit project). There is more potential use of this with the standalone tools concept (#5843). The big picture is also discussed in #5830.



To unsubscribe from these emails, change your notification settings at https://github.com/OSGeo/grass/settings/notifications