W

Ward
GitHub logo
216 stars, 18 forks

Running tests

To find and run tests in your project, you can run ward without any arguments.

$
ward

This will recursively search through the current directory for modules with a name starting with test_, and execute any tests contained in the modules it finds.

Specifying test modules or directories

$
ward --path PATH_TO_TESTS

You can run tests in a specific directory or module using the --path option. For example, to run all tests inside a directory called tests:

$
ward --path tests

To run tests in the current directory, you can just type ward, which is functionally equivalent to ward --path .

You can also directly specify a test module, for example:

$
ward --path tests/api/test_get_user.py

You can supply multiple test directories by providing multiple --path options:

$
ward --path "unit" --path "integration"

Ward will run all tests it finds across all given paths. If one of the specified paths is contained within another, it won't repeat the same test more than once.

Excluding modules or directories

$
ward --exclude glob1 --exclude glob2

You can tell Ward to ignore specific modules or directories using the --exclude command line option. This option can be supplied multiple times, and supports glob patterns.

To configure excluded directories in a more permanent manner, you can use pyproject.toml:

[tool.ward]
exclude = ["glob1", "glob2"]

Tag expressions: Selecting tests with --tags

$
ward --tags EXPR

A tag expression is an infix boolean expression that can be used to accurately select a subset of tests you wish to execute. Tests are tagged using the tags keyword argument of the @test decorator (e.g. @test("eggs are green", tags=["unit"]).)

Here are some examples of tag expressions and what they mean:

Tag ExpressionMeaning
slowtests tagged with slow
unit and integrationtests tagged with both unit and integration
big and not slowtests tagged with big that aren't also tagged with slow
android or iostests tagged with either android or ios

Here's how you would run only tests that are tagged with either android or ios in practice:

$
ward --tags "android or ios"

You can use parentheses in tag expressions to change precedence rules to suit your needs.

Searching: Selecting tests with --search

$
ward --search QUERY

You can choose to limit which tests are collected and ran by Ward using the --search STRING option. Module names, test descriptions and test function bodies will be searched, and those which contain STRING will be ran. Here are some examples:

Run all tests that call the fetch_users function:

$
ward --search "fetch_users("

Run all tests that check if a ZeroDivisionError is raised:

$
ward --search "raises(ZeroDivisionError)"

Run all tests decorated with the @xfail decorator:

$
ward --search "@xfail"

Run a test described as "my_function should return False":

$
ward --search "my_function should return False"

Running tests inside a module:

The search takes place on the fully qualified name, so you can run a single module (e.g. my_module) using the following command:

$
ward --search my_module.

Of course, if a test name or body contains the string "my_module.", that test will also be selected and ran.

This approach is useful for quickly querying tests and running those which match a simple query, making it useful for development.

Customising the output with --test-output-style

$
ward --test-output-style [test-per-line|dots-module|dots-global]

As your project grows, it may become impractical to print each test result on its own line. Ward provides alternative test output styles that can be configured using the --test-output-style option.

test-per-line (default)

The default test output of Ward looks like this (--test-output-style=test-per-line):

PASS test_earth:3:
it is not flat
PASS test_earth:11:
24 hours to rotate
FAIL test_sun:4:
it is hot

dots-module

If you run Ward with --test-output-style=dots-module, each module will be printed on its own line, and a single character will be used to represent the outcome of each test in that module:

tests/test_earth.py: ..
tests/test_sun.py: F

dots-global

If that is still too verbose, you may wish to represent every test outcome with a single character, without grouping them by modules (--test-output-style=dots-global):

..F

Output capturing

By default, Ward captures everything that is written to standard output and standard error as your tests run. If a test fails, everything that was printed during the time it was running will be printed as part of the failure output.

example showing ward capturing output

With output capturing enabled, if you run a debugger such as pdb during test execution, everything it writes to the stdout will be captured by Ward too.

Disabling output capturing

If you wish to disable output capturing you can do so using the --no-capture-output flag on the command line.

$
ward --no-capture-output

You can also disable output capturing using the capture-output key in your pyproject.toml:

[tool.ward]
capture-output = false

Randomise test execution order

$
ward --order "random"

Running tests in a random order can help identify tests that have hidden dependencies on each other. Tests should pass regardless of the order they run in, and they should pass if run in isolation.

Cancelling after a number of failures with --fail-limit

$
ward --fail-limit LIMIT

If you wish for Ward to cancel a run immediately after a specific number of failing tests, you can use the --fail-limit option. To have a run end immediately after 5 tests fail:

$
ward --fail-limit 5

Finding slow running tests

$
ward --show-slowest NUMBER_TO_SHOW

Use --show-slowest 10 to print the 10 slowest tests after the test run completes.

Performing a dry run

$
ward --dry-run

Use --dry-run to simulate a test run. Ward will collect the tests and output as normal, but the tests themselves won't actually run, nor will any fixtures your tests depend on. When using --dry-run, tests will return with an outcome of DRYRUN or SKIP.

This is useful for determining which tests Ward would run if invoked normally.

Outputting to a file (Proposal)

$
ward --output-json PATH

Use --output-json to write test results to a JSON file after the test run completes.

If the file doesn't exist, Ward will create it. If the file does exist, Ward will overwrite it.

Tracking proposal on GitHub: https://github.com/darrenburns/ward/issues/123