Test Framework

The Zephyr Test Framework (Ztest) provides a simple testing framework intended to be used during development. It provides basic assertion macros and a generic test structure.

The framework can be used in two ways, either as a generic framework for integration testing, or for unit testing specific modules.

Quick start - Integration testing

A simple working base is located at samples/testing/integration. Just copy the files to tests/ and edit them for your needs. The test will then be automatically built and run by the sanitycheck script. If you are testing the bar component of foo, you should copy the sample folder to tests/foo/bar. It can then be tested with ./scripts/sanitycheck -s tests/foo/bar/test.

The sample contains the following files:

Makefile

1
2
3
4
BOARD ?= qemu_x86
CONF_FILE ?= prj.conf

include $(ZEPHYR_BASE)/Makefile.inc

testcase.ini

1
2
[test]
tags = my_tags

prj.conf

1
CONFIG_ZTEST=y

src/Makefile

1
2
3
obj-y = main.o

include $(ZEPHYR_BASE)/tests/Makefile.test

src/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
/*
 * Copyright (c) 2016 Intel Corporation
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <ztest.h>

static void assert_tests(void)
{
	assert_true(1, "1 was false");
	assert_false(0, "0 was true");
	assert_is_null(NULL, "NULL was not NULL");
	assert_not_null("foo", "\"foo\" was NULL");
	assert_equal(1, 1, "1 was not equal to 1");
	assert_equal_ptr(NULL, NULL, "NULL was not equal to NULL");
}

void test_main(void)
{
	ztest_test_suite(framework_tests,
		ztest_unit_test(assert_tests)
	);

	ztest_run_test_suite(framework_tests);
}

Quick start - Unit testing

Ztest can be used for unit testing. This means that rather than including the entire Zephyr OS for testing a single function, you can focus the testing efforts into the specific module in question. This will speed up testing since only the module will have to be compiled in, and the tested functions will be called directly.

Since you won’t be including basic kernel data structures that most code depends on, you have to provide function stubs in the test. Ztest provides some helpers for mocking functions, as demonstrated below.

In a unit test, mock objects can simulate the behavior of complex real objects and are used to decide whether a test failed or passed by verifying whether an interaction with an object occurred, and if required, to assert the order of that interaction.

The samples/testing/unit folder contains an example for testing the net-buf api of Zephyr.

Makefile

1
2
3
INCLUDE += subsys

include $(ZEPHYR_BASE)/tests/unit/Makefile.unittest

testcase.ini

1
2
3
[test]
type = unit
tags = buf

main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
/*
 * Copyright (c) 2016 Intel Corporation
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <ztest.h>

unsigned int irq_lock(void)
{
	return 0;
}

void irq_unlock(unsigned int key)
{
}

#include <net/buf.c>

void k_fifo_init(struct k_fifo *fifo) {}
void k_fifo_put_list(struct k_fifo *fifo, void *head, void *tail) {}

int k_is_in_isr(void)
{
	return 0;
}

void *k_fifo_get(struct k_fifo *fifo, int32_t timeout)
{
	return NULL;
}

void k_fifo_put(struct k_fifo *fifo, void *data)
{
}

void k_lifo_init(struct k_lifo *lifo) {}

void *k_lifo_get(struct k_lifo *lifo, int32_t timeout)
{
	return NULL;
}

void k_lifo_put(struct k_lifo *lifo, void *data)
{
}

#define TEST_BUF_COUNT 1
#define TEST_BUF_SIZE 74

NET_BUF_POOL_DEFINE(bufs_pool, TEST_BUF_COUNT, TEST_BUF_SIZE,
		    sizeof(int), NULL);

static void test_get_single_buffer(void)
{
	struct net_buf *buf;

	buf = net_buf_alloc(&bufs_pool, K_NO_WAIT);

	assert_equal(buf->ref, 1, "Invalid refcount");
	assert_equal(buf->len, 0, "Invalid length");
	assert_equal(buf->flags, 0, "Invalid flags");
	assert_equal_ptr(buf->frags, NULL, "Frags not NULL");
}

void test_main(void)
{
	ztest_test_suite(net_buf_test,
		ztest_unit_test(test_get_single_buffer)
	);

	ztest_run_test_suite(net_buf_test);
}

API reference

Running tests

void ztest_test_fail(void)

Fail the currently running test.

This is the function called from failed assertions and the like. You probably don’t need to call it yourself.

static void unit_test_noop(void)

Do nothing, successfully.

Unit test / setup function / teardown function that does nothing, successfully. Can be used as a parameter to ztest_unit_test_setup_teardown().

ztest_unit_test_setup_teardown(fn, setup, teardown) { \ STRINGIFY(fn), fn, setup, teardown \ }

Define a test with setup and teardown functions.

This should be called as an argument to ztest_test_suite. The test will be run in the following order: setup, fn, teardown.

Parameters
  • fn: Main test function
  • setup: Setup function
  • teardown: Teardown function

ztest_unit_test(fn) ztest_unit_test_setup_teardown(fn, unit_test_noop, unit_test_noop)

Define a test function.

This should be called as an argument to ztest_test_suite.

Parameters
  • fn: Test function

ztest_test_suite(name, ...) struct unit_test _##name[] = { \ __VA_ARGS__, { 0 } \ }

Define a test suite.

This function should be called in the following fashion: 

ztest_test_suite(test_suite_name,
        ztest_unit_test(test_function),
        ztest_unit_test(test_other_function)
);

ztest_run_test_suite(test_suite_name);

Parameters
  • name: Name of the testing suite

ztest_run_test_suite(suite) _ztest_run_test_suite(#suite, _##suite)

Run the specified test suite.

Parameters
  • suite: Test suite to run.

Assertions

These macros will instantly fail the test if the related assertion fails. When an assertion fails, it will print the current file, line and function, alongside a reason for the failure and an optional message. If the config option CONFIG_ZTEST_ASSERT_VERBOSE=0, the assertions will only print the file and line numbers, reducing the binary size of the test.

Example output for a failed macro from assert_equal(buf->ref, 2, “Invalid refcount”):

Assertion failed at main.c:62: test_get_single_buffer: Invalid refcount (buf->ref not equal to 2)
Aborted at unit test function
assert(cond, msg, default_msg) _assert(cond, msg ? msg : "", msg ? ("(" default_msg ")") : (default_msg), \ __FILE__, __LINE__, __func__)

Fail the test, if cond is false.

You probably don’t need to call this macro directly. You should instead use assert_{condition} macros below.

Parameters
  • cond: Condition to check
  • msg: Optional, can be NULL. Message to print if cond is false.
  • default_msg: Message to print if cond is false

assert_unreachable(msg) assert(0, msg, "Reached unreachable code")

Assert that this function call won’t be reached.

Parameters
  • msg: Optional message to print if the assertion fails

assert_true(cond, msg) assert(cond, msg, #cond " is false")

Assert that cond is true.

Parameters
  • cond: Condition to check
  • msg: Optional message to print if the assertion fails

assert_false(cond, msg) assert(!(cond), msg, #cond " is true")

Assert that cond is false.

Parameters
  • cond: Condition to check
  • msg: Optional message to print if the assertion fails

assert_is_null(ptr, msg) assert((ptr) == NULL, msg, #ptr " is not NULL")

Assert that ptr is NULL.

Parameters
  • ptr: Pointer to compare
  • msg: Optional message to print if the assertion fails

assert_not_null(ptr, msg) assert((ptr) != NULL, msg, #ptr " is NULL")

Assert that ptr is not NULL.

Parameters
  • ptr: Pointer to compare
  • msg: Optional message to print if the assertion fails

assert_equal(a, b, msg) assert((a) == (b), msg, #a " not equal to " #b)

Assert that a equals b.

a and b won’t be converted and will be compared directly.

Parameters
  • a: Value to compare
  • b: Value to compare
  • msg: Optional message to print if the assertion fails

assert_not_equal(a, b, msg) assert((a) != (b), msg, #a " equal to " #b)

Assert that a does not equal b.

a and b won’t be converted and will be compared directly.

Parameters
  • a: Value to compare
  • b: Value to compare
  • msg: Optional message to print if the assertion fails

assert_equal_ptr(a, b, msg) assert((void *)(a) == (void *)(b), msg, #a " not equal to " #b)

Assert that a equals b.

a and b will be converted to void * before comparing.

Parameters
  • a: Value to compare
  • b: Value to compare
  • msg: Optional message to print if the assertion fails

Mocking

These functions allow abstracting callbacks and related functions and controlling them from specific tests. You can enable the mocking framework by setting CONFIG_ZTEST_MOCKING=y in the configuration file of the test. The amount of concurrent return values and expected parameters is limited by ZTEST_PARAMETER_COUNT.

Here is an example for configuring the function expect_two_parameters to expect the values a=2 and b=3, and telling returns_int to return 5:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#include <ztest.h>

static void expect_two_parameters(int a, int b)
{
	ztest_check_expected_value(a);
	ztest_check_expected_value(b);
}

static void parameter_tests(void)
{
	ztest_expect_value(expect_two_parameters, a, 2);
	ztest_expect_value(expect_two_parameters, b, 3);
	expect_two_parameters(2, 3);
}

static int returns_int(void)
{
	return ztest_get_return_value();
}

static void return_value_tests(void)
{
	ztest_returns_value(returns_int, 5);
	assert_equal(returns_int(), 5, NULL);
}

void test_main(void)
{
	ztest_test_suite(mock_framework_tests,
		ztest_unit_test(parameter_test),
		ztest_unit_test(return_value_test)
	);

	ztest_run_test_suite(mock_framework_tests);
}
ztest_expect_value(func, param, value) _ztest_expect_value(STRINGIFY(func), STRINGIFY(param), \ (uintptr_t)(value))

Tell function func to expect the value value for param.

When using ztest_check_expected_value(), tell that the value of param should be value. The value will internally be stored as an uintptr_t.

Parameters
  • func: Function in question
  • param: Parameter for which the value should be set
  • value: Value for param

ztest_check_expected_value(param) _ztest_check_expected_value(__func__, STRINGIFY(param), \ (uintptr_t)(param))

If param doesn’t match the value set by ztest_expect_value(), fail the test.

This will first check that does param have a value to be expected, and then checks whether the value of the parameter is equal to the expected value. If either of these checks fail, the current test will fail. This must be called from the called function.

Parameters
  • param: Parameter to check

ztest_returns_value(func, value) _ztest_returns_value(STRINGIFY(func), (uintptr_t)(value))

Tell func that it should return value.

Parameters
  • func: Function that should return value
  • value: Value to return from func

ztest_get_return_value _ztest_get_return_value(__func__)

Get the return value for current function.

The return value must have been set previously with ztest_returns_value(). If no return value exists, the current test will fail.

Return
The value the current function should return

ztest_get_return_value_ptr ((void *)_ztest_get_return_value(__func__))

Get the return value as a pointer for current function.

The return value must have been set previously with ztest_returns_value(). If no return value exists, the current test will fail.

Return
The value the current function should return as a void *