[GRASS-git] [OSGeo/grass] a7bb10: grass.script: Use text=True by default for subproc...

[Vaclav Petras]

  Branch: refs/heads/main
  Home:   https://github.com/OSGeo/grass
  Commit: a7bb103537dc5179757a05844431d87018688326
      https://github.com/OSGeo/grass/commit/a7bb103537dc5179757a05844431d87018688326
  Author: Vaclav Petras <wenzeslaus at gmail.com>
  Date:   2025-06-26 (Thu, 26 Jun 2025)

  Changed paths:
    M general/g.mapsets/tests/g_mapsets_list_format_test.py
    M python/grass/gunittest/gmodules.py
    M python/grass/jupyter/utils.py
    M python/grass/pygrass/modules/interface/module.py
    M python/grass/script/core.py
    M python/grass/script/raster.py
    M python/grass/script/task.py
    M python/grass/script/testsuite/test_start_command_functions.py
    M python/grass/script/testsuite/test_start_command_functions_nc.py
    M python/grass/semantic_label/reader.py
    M scripts/i.band.library/testsuite/test_i_band_library.py
    M scripts/m.proj/m.proj.py
    M scripts/r.mask/r.mask.py
    M scripts/v.rast.stats/testsuite/test_v_rast_stats.py
    M temporal/t.rast.gapfill/testsuite/test_gapfill.py
    M temporal/t.rast.univar/testsuite/test_t_rast_univar.py
    M temporal/t.rast.what/testsuite/test_what.py
    M temporal/t.rast3d.univar/testsuite/test_t_rast3d_univar.py

  Log Message:
  -----------
  grass.script: Use text=True by default for subprocesses (#5881)

Unlike in the days of Python 2, Popen has now text mode which makes everything just standard (unicode) strings and newlines just LFs. This PR s trying to switch the default from text=False to text=True deep in grass.script in the Popen class wrapper. The idea is that one can still fallback to bytes and encoding/decoding if needed which is approach taken in grass.script.task for XML. All other places should remove the encoding/decoding code.

Changes in grass.script are mostly in the sense that there should be no impact to the user code. However, some changes are needed in Python tools, suggesting that there is a risk that user code would have to change. The Popen stdin wrapper (see below) allows doing this change with issues a next major release (v9) because we allow for both encoded (bytes) and unicode strings (str) as inputs. Our encode and decode utils functions already work as pass-thru when the input type is the desired output type which helps with the outputs for most cases. Overall, the new code more fits the expectations of "it should just work" and creates least surprise, although it changes the behavior slightly and has a different default than subprocess package (text=True as opposed to text=False). The code which is using it now was adjusted as if the backwards compatibility would not be in place. The places which require a change are limited (see m.proj below).

Wrap Popen's stdin to provide automatic conversion if data passed to write is text encoded in bytes, but the stream is set to text mode (str) automatically. This provides backwards compatibility, so that the old code which encodes explicitly, but does not disable text mode, still works without changes.

Fixes for grass.jupyter and m.proj which will make it work even when we completely switch and remove the compatibility code. m.proj with text=True deadlocks when used from grass.jupyter in any way (explicit stdin or communicate, with or without the stdin wrapper). Using text=False makes m.proj work in this context which makes sense because it is doing a lot of low lever tricks.

For grass.script.task, use bytes and count on bytes in the fallback UTF-8 encoding procedure (bytes don't have encode), and try harder to decode the received bytes.

In grass.pygrass and grass.gunittest, do not encode before passing to subprocess.Popen. grass.pygrass.modules already encodes/decodes for the user. grass.gunittest call_module allowed both, but was passing bytes to Popen.

Update the start_command tests which tested specifically with bytes to str. Test specifically for bytes as input separately.

Avoid relying on os.linesep and rely on higher level code with text=True to avoid breaking the test on Windows in i.band.library. For printing semantic labels, use print and f-strings instead of sys.stdout.write to handle different cases which handles newline characters automatically.

Remove xfail_windows from tests which are now running on Windows.



To unsubscribe from these emails, change your notification settings at https://github.com/OSGeo/grass/settings/notifications