ci_exec.core

The core functionality of the ci_exec package.

fail(why, *[, exit_code, no_prefix])

Write a failure message to sys.stderr and exit.

Executable(exe_path, *[, log_calls, …])

Represent a reusable executable.

mkdir_p(path[, mode, parents, exist_ok])

Permissive wrapper around pathlib.Path.mkdir().

rm_rf(path[, ignore_errors, onerror])

Permissive wrapper around shutil.rmtree() bypassing FileNotFoundError and NotADirectoryError.

which(cmd, *[, mode, path])

Restrictive wrapper around shutil.which() that will fail() if not found.
fail(why, *, exit_code=1, no_prefix=False)[source]

Write a failure message to sys.stderr and exit.

Parameters

  • why (str) – The message explaining why the program is being failed out.

  • exit_code (int) – The exit code to use. Default: 1.

  • no_prefix (bool) – Whether to prefix a bold red "[X] " before why. Default: False, the bold red "[X] " prefix is included unless set to True.

class Executable(exe_path, *, log_calls=True, log_prefix='$ ', log_color='36', log_style='1')[source]

Represent a reusable executable.

Each executable is:

  1. Failing by default: unless called with check=False, any execution that fails (has a non-zero exit code) will result in a call to fail(), terminating the entire application.

  2. Logging by default: every call executed will print what will be run in color and then dump the output of the command. In the event of failure, this makes finding the last call issued much simpler.

Consider the following simple script:

from ci_exec import Executable
git = Executable("/usr/bin/git")
git("remote")
# Oops! --pretty not --petty ;)
git("log", "-1", "--petty=%B")
git("status")  # will not execute (previous failed)

When we execute python simple.py and check the exit code with echo $?:

> python simple.py
$ /usr/bin/git remote
origin
$ /usr/bin/git log -1 --petty=%B
fatal: unrecognized argument: --petty=%B
[X] Command '('/usr/bin/git', 'log', '-1', "--petty=%B")' returned
    non-zero exit status 128.
> echo $?
128

See __call__() for more information.

Tip

Hard-coded paths in these examples were for demonstrative purposes. In practice this should not be done, use which() instead.

exe_path

The path to the executable that will be run when called.

Type

str

log_calls

Whether or not every invocation of __call__() should print what will execute before executing it. Default: True.

Type

bool

log_prefix

The prefix to use when printing a given invocation of __call__(). Default: "$ " to simulate a console lexer. Set to the empty string "" to have no prefix.

Type

str

log_color

The color code to use when calling colorize() to display the next invocation of __call__(). Set to None to disable colorizing each log of __call__(). Default: Colors.Cyan.

Type

str

log_style

The style code to use when calling colorize() to display the next invocation of __call__(). If no colors are desired, set log_color to None. Default: Styles.Bold.

Type

str

Raises

ValueError – If exe_path is not a file, or if it is not executable.

PATH_EXTENSIONS = {}

The set of valid file extensions that can be executed on Windows.

On *nix systems this will be the empty set, and takes no meaning. On Windows, it is controlled by the user. These are stored in lower case, and comparisons should be lower case for consistency. The typical default value on Windows would be:

PATH_EXTENSIONS = {".com", ".exe", ".bat", ".cmd"}
__call__(*args, **kwargs)[source]

Run exe_path with the specified command-line *args.

The usage of the parameters is best summarized in code:

popen_args = (self.exe_path, *args)
# ... some potential logging ...
return subprocess.run(popen_args, **kwargs)

For example, sending multiple arguments to the executable is as easy as:

cmake = Executable("/usr/bin/cmake")
cmake("..", "-G", "Ninja", "-DBUILD_SHARED_LIBS=ON")

and any overrides to subprocess.run() you wish to include should be done with **kwargs, which are forwarded directly.

Warning

Any exceptions generated result in a call to fail(), which will terminate the application.

Parameters

  • *args – The positional arguments will be forwarded along with exe_path to subprocess.run().

  • **kwargs –

    The key-value arguments are all forwarded to subprocess.run(). If check is not provided, this is an implicit check=True. That is, if you do not want the application to exit (via fail()), you must specify check=False:

    >>> from ci_exec import Executable
>>> git = Executable("/usr/bin/git")
>>> proc = git("not-a-command", check=False)
$ /usr/bin/git not-a-command
git: 'not-a-command' is not a git command. See 'git --help'.
>>> proc.returncode
1
>>> git("not-a-command")
$ /usr/bin/git not-a-command
git: 'not-a-command' is not a git command. See 'git --help'.
[X] Command '('/usr/bin/git', 'not-a-command')' returned non-zero exit
    status 1.

    The final git("not-a-command") exited the shell (this is what is meant by “failing by default”).

Returns

The result of calling subprocess.run() as outlined above.

Note

Unless you are are calling with check=False, you generally don’t need to store the return type.

Return type

subprocess.CompletedProcess

mkdir_p(path, mode=511, parents=True, exist_ok=True)[source]

Permissive wrapper around pathlib.Path.mkdir().

The intention is to behave like mkdir -p, meaning the only real difference is that parents and exist_ok default to True for this method (rather than False for pathlib).

Parameters

  • path (pathlib.Path or str) – The directory path to make.

  • mode (int) – Access mask for directory permissions. See pathlib.Path.mkdir().

  • parents (bool) – Whether or not parent directories may be created. Default: True.

  • exist_ok (bool) –

    Whether or not the command should be considered successful if the specified path already exists. Default: True.

    Note

    If the path exists and is a directory with exist_ok=True, the command will succeed. If the path exists and is a file, even with exist_ok=True the command will fail().

rm_rf(path, ignore_errors=False, onerror=None)[source]

Permissive wrapper around shutil.rmtree() bypassing FileNotFoundError and NotADirectoryError.

This function simply checks if path exists first before calling shutil.rmtree(). If the path does not exist, nothing is done. If the path exists but is a file, pathlib.Path.unlink() is called instead.

Essentially, this function tries to behave like rm -rf, but in the event that removal is not possible (e.g., due to insufficient permissions), the function will still fail().

Parameters

  • path (pathlib.Path or str) – The directory path to delete (including all children).

  • ignore_errors (bool) – Whether or not errors should be ignored. Default: False, to ensure that permission errors are still caught.

  • onerror – See shutil.rmtree() for more information on the callback.

which(cmd, *, mode=1, path=None, **kwargs)[source]

Restrictive wrapper around shutil.which() that will fail() if not found.

The primary difference is that when cmd is not found, shutil.which() will return None whereas this function will fail(). If you need to conditionally check for a command, do not use this function, use shutil.which() instead.

Parameters

  • cmd (str) – The name of the command to search for. E.g., "cmake".

  • mode (int) – The flag permission mask. Default: (os.F_OK | os.X_OK), see: os.F_OK, os.X_OK, shutil.which().

  • path (str or None) – Default: None. See shutil.which().

  • **kwargs –

    Included as a convenience bypass, forwards directly to Executable constructor. Suppose a non-logging Executable is desired. One option:

    git = which("git")
git.log_calls = False

    Or alternatively:

    git = which("git", log_calls=False)

    This is in recognition that for continuous integration users will likely have many different preferences. Users can provide their own which to always use this default, or say, change the logging color:

    from ci_exec import which as ci_which
from ci_exec import Colors, Styles

def which(cmd: str):
    return ci_which(cmd, log_color=Colors.Magenta, log_style=Styles.Regular)

Returns

An executable created with the full path to the found cmd.

Return type

Executable

Tests

Tests for the ci_exec.core module.

test_fail(capsys, why, exit_code, no_prefix)[source]

Validate fail() exits as expected.

test_executable_construction_failures()[source]

Validate that non-absolute and non-(executable)file constructions will raise.

test_executable_relative()[source]

Validate Executable accepts relative paths.

test_executable_logging(capsys)[source]

Validate Executable runs and logs as expected.

test_executable_failures(capsys)[source]

Validate failing executables error as expected.

test_mkdir_p(capsys)[source]

Validate that mkdir_p() creates directories as expected.

test_rm_rf(capsys)[source]

Validate rm_rf() deletes files / directories as expected.

test_which(capsys)[source]

Validate that which() finds or does not find executables.