W

Ward
GitHub logo
184 stars, 12 forks

Running tests

To find and run tests in your project, you can run ward without any arguments.

$
ward

This will recursively search through the current directory for modules with a name starting with test_, and execute any tests contained in the modules it finds.

Specifying test modules or directories

$
ward --path PATH_TO_TESTS

You can run tests in a specific directory or module using the --path option. For example, to run all tests inside a directory called tests:

$
ward --path tests

To run tests in the current directory, you can just type ward, which is functionally equivalent to ward --path .

You can also directly specify a test module, for example:

$
ward --path tests/api/test_get_user.py

You can supply multiple test directories by providing multiple --path options:

$
ward --path "unit" --path "integration"

Ward will run all tests it finds across all given paths. If one of the specified paths is contained within another, it won't repeat the same test more than once.

Excluding modules or directories

$
ward --exclude glob1 --exclude glob2

You can tell Ward to ignore specific modules or directories using the --exclude command line option. This option can be supplied multiple times, and supports glob patterns.

To configure excluded directories in a more permanent manner, you can use pyproject.toml:

[tool.ward]
exclude = ["glob1", "glob2"]

Customising the output with --test-output-style

$
ward --test-output-style [test-per-line|dots-module|dots-global]

As your project grows, it may become impractical to print each test result on its own line. Ward provides alternative test output styles that can be configured using the --test-output-style option.

test-per-line (default)

The default test output of Ward looks like this (--test-output-style=test-per-line):

PASS test_earth:3:
it is not flat
PASS test_earth:11:
24 hours to rotate
FAIL test_sun:4:
it is hot

dots-module

If you run Ward with --test-output-style=dots-module, each module will be printed on its own line, and a single character will be used to represent the outcome of each test in that module:

tests/test_earth.py: ..
tests/test_sun.py: F

dots-global

If that is still too verbose, you may wish to represent every test outcome with a single character, without grouping them by modules (--test-output-style=dots-global):

..F

Approximate test selection with --search

$
ward --search QUERY

You can choose to limit which tests are collected and ran by Ward using the --search STRING option. Test names, descriptions and test function bodies will be searched, and those which contain STRING will be ran. Here are some examples:

Run all tests that call the fetch_users function:

$
ward --search "fetch_users("

Run all tests that check if a ZeroDivisionError is raised:

$
ward --search "raises(ZeroDivisionError)"

Run all tests decorated with the @xfail decorator:

$
ward --search "@xfail"

Run a test described as "my_function should return False":

$
ward --search "my_function should return False"

Running tests inside a module:

The search takes place on the fully qualified name, so you can run a single module (e.g. my_module) using the following command:

$
ward --search my_module.

Of course, if a test name or body contains the string "my_module.", that test will also be selected and ran.

This approach is useful for quickly querying tests and running those which match a simple query, making it useful for development.

Specific test selection

Sometimes you want to be very specific when declaring which tests to run.

Ward will provide an option to query tests on name and description using substring or regular expression matching.

(TODO)

Output capturing

By default, Ward captures everything that is written to standard output and standard error as your tests run. If a test fails, everything that was printed during the time it was running will be printed as part of the failure output.

example showing ward capturing output

With output capturing enabled, if you run a debugger such as pdb during test execution, everything it writes to the stdout will be captured by Ward too.

Disabling output capturing

If you wish to disable output capturing you can do so using the --no-capture-output flag on the command line.

$
ward --no-capture-output

You can also disable output capturing using the capture-output key in your pyproject.toml:

[tool.ward]
capture-output = false

Randomise test execution order

$
ward --order "random"

Running tests in a random order can help identify tests that have hidden dependencies on each other. Tests should pass regardless of the order they run in, and they should pass if run in isolation.

Cancelling after a number of failures with --fail-limit

$
ward --fail-limit LIMIT

If you wish for Ward to cancel a run immediately after a specific number of failing tests, you can use the --fail-limit option. To have a run end immediately after 5 tests fail:

$
ward --fail-limit 5