Important Usage Notes¶
Namespace Pollution¶
The ci_exec
package namespace is polluted intentionally. Always import from the
polluted namespace, never import from the module originally defining something:
from ci_exec import which # Yes
from ci_exec.core import which # NO!
In practice it shouldn’t matter that much, but
Any functions moved to different modules will not be considered ABI breaking changes. So if
ci_exec.core.which
moved to a differentci_exec.other_module
, this would be done in a patch release (ci_exec
uses semantic versioning).Anything that is not exposed at the top-level is more than likely something that should not be used.
Beware of Commands Expecting Input¶
The Executable.__call__()
method internally
invokes subprocess.run()
, which does some fancy manipulations of
stdout
, stderr
, and stdin
. In particular, the subprocess by default
will allow communication on stdin
for commands that are expecting input. In the
context of CI, this is problematic, and users need to be aware that the subprocess will
indeed pause and solicit user input. Consider a scenario where say filter_file()
was
used to make some significant changes on the CI provider. If you ran:
from ci_exec import which
git = which("git")
git("diff")
The build may fail from a timeout rather than just displaying the changes, because for a
large enough change to report git
will want to use your $PAGER
to let you
scroll through it. Most commands that allow input also provide a command-line flag to
disable this behavior and in this scenario, it would be:
git("--no-pager", "diff")
In situations where you know that input is required, subprocess.run()
enables this via the input
parameter. Consider the following toy script that is
expecting two pieces of user input at different stages:
name = input("What is your name? ")
print(f"Hi {name}, nice to meet you.")
age = input("How old are you? ")
print(f"Wow! {age} is such a great age :)")
Then you could call it using ci_exec
like this:
import sys
from ci_exec import Executable
if __name__ == "__main__":
python = Executable(sys.executable)
python("./multi_input.py", input=b"Bilbo\n111")
where
The
input
parameter needs abytes
object (theb
prefix inb"..."
).You use
\n
to send a newline. From the docs: For stdin, line ending characters'\n'
in the input will be converted to the default line separatoros.linesep
. So"Bilbo\n"
will end up being the answer to the firstname = input(...)
, and"111"
will be sent to theage = input(...)
.
Build Logs out of Order (Azure Pipelines)¶
If the results of call logging appear out of order, then your CI provider is affected by this. This is a known problem on Azure Pipelines, it seems unlikely to affect many other providers due to the nature of how the Azure Pipelines build logs are served. Example “script”:
cmake("..")
ninja()
Produces:
-- The C compiler identification is GNU 8.3.1
-- The CXX compiler identification is GNU 8.3.1
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
...
[12/74] Building C object ...joystick.c.o
$ /usr/bin/cmake ..
$ /usr/bin/ninja
The log is out of order, all cmake / ninja output appeared before the call logging
($ /usr/bin/cmake ..
and $ /usr/bin/ninja
). There are two possible solutions:
Invoke your build script using python -u:
python -u ./.ci/build.py
Set the environment variable PYTHONUNBUFFERED=true.