[gdal-dev] Standardize gdal-utils scripts return code for "no arguments"

Tue Apr 26 13:05:50 PDT 2022

Here is the path I'm following in trying to deconstruct GDALArgumentParser.
With file = pc2rgb.py:
        r = subprocess.run([sys.executable,
            file],
            shell=True,
            capture_output=True,
            text=True,
            )
        if debug:
            print(f'returncode: {r.returncode}')
I get:
returncode: 2
Digging in to pct2rgb.py I see main() calling PCT2RGB class, calling pct2rgb function, which should error out with an exception.:
def main(argv=sys.argv):
    return PCT2RGB().main(argv)
    def doit(self, **kwargs):
        return pct2rgb(**kwargs)
def doit(**kwargs):
    try:
        ds = pct2rgb(**kwargs)
        return ds, 0
    except:
        return None, 1
def pct2rgb(src_filename: PathLikeOrStr, pct_filename: Optional[PathLikeOrStr], dst_filename: PathLikeOrStr,
            band_number: int = 1, out_bands: int = 3, driver_name: Optional[str] = None):
    # Open source file
    src_ds = open_ds(src_filename)
    if src_ds is None:
        raise Exception(f'Unable to open {src_filename} ')
    ...
So I'm thinking that while PCT2RGB class is invoking GDALArgumentParser it's exiting early somehow with this 2 return code.
def get_parser(self, argv) -> GDALArgumentParser:
    ...
    return parser
I look at auxillary/gdal_argparse.py: class GDALArgumentParser(argparse.ArgumentParser) and I see... well a bunch of stuff I only apprehend the shadowy outlines of. Looking at the return statements doesn't add light. (It doesn't help that I don't understand how classes work. There's this thing called super() but it's not defined anywhere. Shouldn't that cause an error? Oh it's from stdlib builtins.pyi. What the heck is a pyi? "...It is pitch black. You are likely to be eaten by a grue.")
> findstr "return" ..\auxiliary\gdal_argparse.py
        return super().parse_args(args=args, **kwargs)
        return shlex.split(arg_line, comments=True)
        return self._parser
        return kwargs
        return kwargs
            return 0
            return 1
        return epilog or None
So my question is: where is returncode: 2 coming from?

-Matt

From: Matt.Wilkie
Sent: April 26, 2022 1:03 PM
To: 'gdal-dev at lists.osgeo.org' <gdal-dev at lists.osgeo.org>
Subject: RE: Standardize gdal-utils scripts return code for "no arguments"

Hi Folks,

I've converted all the scripts that were using -1 to 1. However when I started looking at the ones returning 2 it became less clear what to do. Excepting gdal2tiles all of them are using GDALArgumentParser and I don't see where the return value is being set. Thinking this might mean the mainline utils might be using 2 for no args I checked gdal_translate and gdalwarp, but no, both of those use 1.

The scripts that return 2 are:

osgeo_utils\gdal2xyz.py
osgeo_utils\gdal_calc.py
osgeo_utils\gdal_fillnodata.py
osgeo_utils\gdal_polygonize.py
osgeo_utils\pct2rgb.py
osgeo_utils\rgb2pct.py
osgeo_utils\samples\gdallocationinfo.py

I haven't been able to figure out the GDALArgumentParser  parser class so I embarked on making everything else use 2 instead of 1. I'm questioning the wisdom of that at the moment since so many files are touched. However the process has forced me to look more closely at the many scripts and start to internalize the various patterns they use. This has been worthwhile even if the approach might get abandoned. The "make everything return 2" effort is in branch patch-5561-ret2<https://github.com/maphew/gdal/tree/patch-5561-ret2>.

The next message will have more details on how I've tried and failed to understand GDALArgumentParser.

-Matt

From: gdal-dev <gdal-dev-bounces at lists.osgeo.org<mailto:gdal-dev-bounces at lists.osgeo.org>> On Behalf Of Matt.Wilkie at yukon.ca<mailto:Matt.Wilkie at yukon.ca>
Sent: April 4, 2022 4:07 PM
To: gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
Subject: [gdal-dev] Standardize gdal-utils scripts return code for "no arguments"

Hi folks,

I'm working on "Standardize gdal-utils scripts return codes #5561" for all the scripts in swig/python/gdal-utils. Currently the scripts do not return the same status code for "was run without arguments".

It would be good for the same code meant the same thing across all the scripts in the package. Given that most of the scripts use `1` now, and that this is in line with sys,exit() docs I think it makes sense to make 1 the new standard.

At present we have (return_code, num scripts with that code):

0: 30
1: 56
2:  8
-1:  9

-1 is a special case. In the script code it's written as `return -1` but subprocess.run() captures it as `4294967295`. Another special case is `gdal_auth.py` sample which with no arguments spawns a web authentication page in browser.

Changing the return code will mean anyone who is relying on those in their own scripts or programs will need to adjust. This seems to be a small price for the gain in harmonization across the utilities, to me. Your thoughts?

[0]: https://github.com/OSGeo/gdal/issues/5561<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgdal%2Fissues%2F5561&data=04%7C01%7Cmatt.wilkie%40yukon.ca%7Cc3e8dba890e54a879a4808da168fdceb%7C98f515313973490abb70195aa264a2bc%7C0%7C0%7C637847105002756645%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=cJaW4TxL68DJMI8KTEjVjQCIzsN1Oa9ShUtILmy1%2B2g%3D&reserved=0>
[1]: https://docs.python.org/3/library/sys.html#sys.exit<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.python.org%2F3%2Flibrary%2Fsys.html%23sys.exit&data=04%7C01%7Cmatt.wilkie%40yukon.ca%7Cc3e8dba890e54a879a4808da168fdceb%7C98f515313973490abb70195aa264a2bc%7C0%7C0%7C637847105002756645%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pHA7JH0IS7tjePzMXO1MzvO6svuPVHVR9uQgPqxOtMY%3D&reserved=0>, "Unix programs generally use 2 for command line syntax errors and 1 for all other kind of errors"

Matt Wilkie
Geomatics Developer & Administrator
Environment | Technology, Innovation and Mapping
T 867-667-8133 | Yukon.ca<https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fyukon.ca%2F&data=04%7C01%7Cmatt.wilkie%40yukon.ca%7Cc3e8dba890e54a879a4808da168fdceb%7C98f515313973490abb70195aa264a2bc%7C0%7C0%7C637847105002756645%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=S08L%2BQsPMBZOc2%2Bb3wnWS8W08a0ejZpHRC1QAMfFEyk%3D&reserved=0>
Hours: 08:30-16:30, Mon-Wed: Office, Thu: Remote, Fri: Away.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220426/66bab844/attachment-0001.html>