Getting Started

Warning

This is the old tbot introduction. The new version can be found at

First Steps

tbot works out of the box. You can run it’s selftests like this:

~$ tbot selftest

(If this does not work, please contact the developers. This should not be the case)

Now, let’s create some example testcase. Start by creating a file named tc.py in your current directory. Later you will see that you can also name it differently or have multiple files, but for now, this is the easiest way. Add the following content:

1
2
3
4
5
6
7
import tbot

@tbot.testcase
def my_awesome_testcase() -> None:
    with tbot.acquire_lab() as lh:
        name = lh.exec0("uname", "-n").strip()
        tbot.log.message(f"Hello {name}!")

If you did everything correctly, running

~$ tbot my_awesome_testcase

should greet you host. Let’s disect the code from above:

1
2
3
@tbot.testcase
def my_awesome_testcase() -> None:
    ...

This is just a normal python function for our testcase. The tbot.testcase() decorator tells tbot that it should be treated as a testcase. In practice this means that tbot will allow calling it from the commandline and will hook into the call so it can gather log data about it.

with tbot.acquire_lab() as lh:
    ...

To understand this line, we first need to get to know one of the core concepts of tbot: Machines (Machine). Every host tbot interacts with is called a machine. That includes the labhost, which we use here, a buildhost where your code might be compiled, the board you are testing, or any other host you want tbot to connect to. There are different kinds of machines. Our labhost is special, because it is the base from where connections to other host are made.

Machines should always be used inside a with statement to ensure proper cleanup in any case. This is especially important with boardmachines, because if this is not done, the board might not be turned off after the tests.

The line you see here requests a new labhost object from tbot so we can interact with it. As you will see later, this is not quite the way you would do this normally, but for this simple example it is good enough.

name = lh.exec0("uname", "-n").strip()

Now that we have the ability to interact with the labhost, let’s do so: We call uname -n to greet the users machine. Note, that each argument is passed separately to exec0(). The reason for this is that it ensures everything will be properly escaped and there are no accidental mistakes. For special characters there is a different notation as you will see later.

The strip() is needed, because the command output contains the trailing newline, which we don’t want in this case.

tbot.log.message(f"Hello {name}!")

tbot.log.message() is basically tbot’s equivalent of print(). The most important difference is, that it does not only print it to the terminal, but also store it in the logfile.

Note

tbot has different Verbosity levels:

  • QUIET: Only show testcases that are called

  • INFO: Show info messages, such as those created by tbot.log.message()

  • COMMAND: Show all commands that are called on the various machine

  • STDOUT: Also show commands outputs

  • CHANNEL: Show everything received on all channels, useful for debugging

The default is INFO. You can increase the Verbosity using -v and decrease it using -q.

Writing Testcases

As mentioned above, testcases calling tbot.acquire_lab() is not the best way to do it. Why? Well, imagine, each testcase that is called would create a new ssh connection to your labhost. This would be really inefficient. The easiest solution is to require the lab as a parameter like this:

1
2
3
4
5
6
import tbot
from tbot.machine import linux

@tbot.testcase
def my_testcase(lab: linux.LabHost) -> None:
    ...

This has the big disadvantage that a testcase like this can’t be called from tbot’s commandline, because where would it get that parameter from?

The solution is a hybrid and looks like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import typing
import tbot
from tbot.machine import linux

@tbot.testcase
def my_testcase(
    lab: typing.Optional[linux.LabHost] = None,
) -> None:
    with lab or tbot.acquire_lab() as lh:
        name = lh.exec0("uname", "-n").strip()
        tbot.log.message(f"Hello {name}!")

This is one of my ‘recipes’. These are code snippets that you will reuse all the time while using tbot. There are a lot more, for different tasks. Take a look at the Recipes page.

Note

In this documentation and in the tbot sources, type annotations are used everywhere. This allows the use of a static type-checker such as mypy, which makes finding bugs before you even run the code a lot easier. Of course, this is optional, the following code would work just as well:

1
2
3
4
5
6
7
import tbot

@tbot.testcase
def my_testcase(lab = None):
    with lab or tbot.acquire_lab() as lh:
        name = lh.exec0("uname", "-n")
        tbot.log.message(f"Hello {name}!")

Calling other testcases is just as easy as calling a python function. From your perspective, a testcase is just a python function. If you want to call testcases from other files, import them like you would with a python module.

tbot contains a library of testcases for common tasks that you can make use of. Take a look at tbot.tc.

Machine Interaction

Linux

All LinuxMachine implement three methods for executing commands: exec(), exec0(), and test(). exec0() is just a wrapper around exec() that ensures the return code of the command is 0. test() returns True if the command finished with return code 0 and False otherwise. Both take the command as one argument per commandline parameter. For example:

1
2
3
output = m.exec0("uname", "-n")
output = m.exec0("dmesg")
output = m.exec0("echo", "$!#?")

tbot will ensure that arguments are properly escaped, so you can pass in anything without worrying. This poses a problem, when you need special syntaxes. For example when you try to pipe the output of one command into another command. To do this in tbot, use code like the following:

1
2
3
from tbot.machine import linux

usb_msg = m.exec0("dmesg", linux.Pipe, "grep", "usb")

This is not the only special parameter you can use:

  • Pipe: A | for piping command output to another command

  • Then: A ; for running multiple commands

  • Background: A & for running a command in the background

  • AndThen: A && for chaining commands

  • OrElse: A || for error handling

There are even more, for more complex use cases:

  • F(): Format string, for complex argument construction. Generally, you won’t need this, because you can just pass each parameter separately. An example, where F() is needed is a parameter that contains a path. Eg:

    # Add a path to $PATH
m.exec0("export", linux.F("PATH={}:{}", mypath, m.env("PATH")))

    What happens here? First of all, m.env("PATH") retrieves the current path. Then, we use linux.F and a format string to create the parameter. You can’t use an f-String in this case, because you can’t trivially turn a tbot path into a string.

  • Env(): Environment variable expansion. Sometimes you want to give an environment variable as a parameter. You can use linux.Env for exactly that. Example:

    m.exec0("echo", "Compiler:", linux.Env("CC"))

    This isn’t the best way to do it, though. I highly reccomend using the following code instead:

    m.exec0("/bin/ls", "-1", m.env("HOME"))

    env() will retrieve the value of the environment variable and return it as a string. The benefit of doing it this way is, that the value will be visible in the logfile and can be read when debugging a failure later on. If you use linux.Env, the log (and tbot) will never actually see the value of the environment variable and you can only guess what it was.

  • Raw(): Raw string if tbot isn’t expressive enough for your usecase. Use this only when no other option works.

Another thing tbot handles specially is paths. A Path can be created like this:

1
2
3
from tbot.machine import linux

p = linux.Path(machine, "/foo/bar")

p is now a Path. tbot’s paths are based on python’s pathlib so you can use all the usual methods / operators:

1
2
3
4
5
file_in_p = p / "dirname" / "file.txt"
if not p.exists():
    ...
if not p.is_dir():
    raise RuntimeError(f"{p} must be a directory!")

tbot’s paths have a very nice property: They are bound to the host they were created with. This means that you cannot accidentally use a path on a wrong machine:

1
2
3
4
5
6
7
m = tbot.acquire_lab()
lnx = tbot.acquire_linux(...)

p = linux.Path(m, "/path/to/somewhere/file.txt")

# This will raise an Exception and will be catched by a static typechecker like mypy:
content = lnx.exec0("cat", p)

Board

Interacting with the board is similar to interacting with a host like the labhost. The only difference is that this time, we need to first initialize the board:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
with tbot.acquire_board(lh) as b:
    with tbot.acquire_uboot(b) as ub:
        ub.exec0("version")

        # Now boot into Linux
        with tbot.acquire_linux(ub) as lnx:
            lnx.exec0("uname", "-a")


# You can also boot directly into Linux:
# (Some boards might not even support intercepting
# U-Boot first)
with tbot.acquire_board(lh) as b:
    with tbot.acquire_linux(b) as lnx:
        lnx.exec0("uname", "-a")

Note

A pattern similar to the one above can be used to write testcases that can either be used from the commandline or supplied with a board-machine:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
import contextlib
import typing
import tbot
from tbot.machine import board


@tbot.testcase
def my_testcase(
    lab: typing.Optional[tbot.selectable.LabHost] = None,
    uboot: typing.Optional[board.UBootMachine] = None,
) -> None:
    with contextlib.ExitStack() as cx:
        lh = cx.enter_context(lab or tbot.acquire_lab())
        if uboot is not None:
            ub = uboot
        else:
            b = cx.enter_context(tbot.acquire_board(lh))
            ub = cx.enter_context(tbot.acquire_uboot(b))

        ...

Again, take a look at the Testcase with U-Boot section on the Recipes page.

Interactive

One convenience function of tbot is allowing the user to directly access most machines’ shells. There are two ways to do so.

  1. Calling one of the interactive_lab, interactive_build, interactive_board, interactive_uboot interactive_linux testcases. This is the most straight forward. It might look like this:

    ~$ tbot -l labs/mylab.py -b boards/myboard.py interactive_uboot

  1. Calling machine.interactive() in your testcase. For example:

    1
2
3
4
5
6
7
8
9
    with tbot.acquire_board(lh) as b:
    with tbot.acquire_linux(b) as lnx:
        lnx.exec0("echo", "Doing some setup work")

        # Might raise an Exception if tbot was not able to reaquire the shell after
        # the interactive session
        lnx.interactive()

        lnx.exec0("echo", "Continuing testcase after the user made some adjustments")