Building HDL and Running Tests

Added in version 1.8.

Warning

Python runners and associated APIs are an experimental feature and subject to change.

The Python Test Runner (short: “runner”) described here is a replacement for cocotb’s traditional Makefile flow. It builds the HDL for the simulator and runs the tests.

It is not meant to be the ideal solution for everyone. You can easily integrate cocotb into your custom flow, see Extending Existing Build Flows.

Usage with pytest

The runner can be used with pytest which is Python’s go-to testing tool.

For an example on how to set up the runner with pytest, see the file cocotb-root/examples/simple_dff/test_dff.py, with the relevant part shown here:

def test_simple_dff_runner():
    sim = os.getenv("SIM", "icarus")

    proj_path = Path(__file__).resolve().parent

    if LANGUAGE == "verilog":
        sources = [proj_path / "dff.sv"]
    else:
        sources = [proj_path / "dff.vhdl"]

    runner = get_runner(sim)
    runner.build(
        sources=sources,
        hdl_toplevel="dff",
        always=True,
    )

    runner.test(hdl_toplevel="dff", test_module="test_dff,")

You run this file with pytest like

SIM=questa HDL_TOPLEVEL_LANG=vhdl pytest examples/simple_dff/test_dff.py

Note that the environment variables SIM and HDL_TOPLEVEL_LANG are defined in this test file to set arguments to the runner’s Runner.build and Runner.test functions; they are not directly handled by the runner itself.

Test filenames and functions have to follow the pytest discovery convention in order to be automatically found.

By default, pytest will only show you a terse “pass/fail” information. To see more details of the simulation run, including the output produced by cocotb, add the -s option to the pytest call:

SIM=questa HDL_TOPLEVEL_LANG=vhdl pytest examples/simple_dff/test_dff.py -s

Note

Take a look at the list of command line flags in pytest’s official documentation.

Direct usage

You can also run the test directly, that is, without using pytest, like so

python examples/simple_dff/test_dff.py

This requires that you define the test to run in the testcase file itself. For example, add the following code at the end:

if __name__ == "__main__":
    test_simple_dff_runner()

Generate waveforms

For many simulators it is possible to generate simulation waveforms. This can be done by setting the waves argument to True in the Runner.build and Runner.test functions. It is also possible to set the environment variable WAVES to a non-zero value to generate waveform files at run-time without modifying the test code, e.g.,

WAVES=1 pytest examples/simple_dff/test_dff.py

Similarly, it is possible to disable the waveform generation set in the test code by setting WAVES to 0.

Starting in GUI mode/viewing waveforms

To start the simulator GUI or, for simulators not having a GUI, view the waveform file in a waveform viewer after the simulation, it is possible to set the gui argument to True in Runner.test. It is also possible to set the GUI environment variable to a non-zero value, e.g.,

GUI=1 pytest examples/simple_dff/test_dff.py

For simulators without a GUI, cocotb will open a waveform viewer. Either Surfer or GTKWave <https://gtkwave.github.io/gtkwave/>, in that order, if available in the system path. To specify preferred waveform viewer, the COCOTB_WAVEFORM_VIEWER environment variable can be used. This can also be used to set, e.g., the surfer.sh startup script for WSL.

API

The API of the Python runner is documented in section Python Test Runner.