Btrax - an automated coverage test tool -
This presentation describes Btrax (Branch TRAcer for linuX), automated C/C++ source code coverage test tools which based on Branch Trace Store (BTS) function in IA32/Intel64 CPU.
Btrax can show C/C++ source code line level coverage, basic block coverage of the binary and assembler level execution path. But, Btrax does not require any source code modification, re-linkage to special libraries or recompilation. Btrax only requires debug information (debuginfo), which is provided by gcc compiler with debug option (-g).
Btrax can be applied to Linux kernel itself, kernel modules and user-land applications. And Btrax can control the coverage analysis range (e.g. from a function entry to exit) for reducing runtime overhead.
Btrax provides great helps developers by graphically showing the covered line and non-covered line in each test case or sum of selected test cases. Btrax also shows the statistics such as function coverage ratio, line coverage ratio or basic block coverage ratio (this can be thought as C0-level coverage).
Btrax can also be used as an assembler level tracer by using execution path view.
Source code coverage test is the test that tries to make every single line of source code to run at least once. This is very fundamental test for ensuring the quality of the software. But it sometimes requires very high testing cost especially for kernel level software.
Before Btrax, there are two major methods for automated coverage test. One is modifying the source code by tools like gcov. This one is very useful but less overhead. The problems of this way are only applicable for user land application and that tested binary is not exactly what is really used. The other method is to make the target application run in single step mode and collect the jmp/call data (or use emulator to collect). The problem of this method is heavy overhead.
The basic architecture of Btrax follows. Btrax is consisting of kernel modules (btrax module), runtime daemon (btraxd) and user land log analysis tools (coverage viewer and execution path viewer). Btrax also provides wrapper script if users use it for application test.
BTS function in IA32/Intel64 collects all jmp/call instruction that is executed. The collected data contains source address, destination address and flags. Execution path viewer can reproduce the execution path by using these data and its binary. To show the source code coverage, Coverage viewer divides binary to "basic block", which is the minimal unit of binary that is not divided by jmp or call instruction. And Coverage viewer colors each basic block whether executed or not-executed. Then it translates each basic block to the line number of its source code by analyzing debuginfo by using elfutils and binutils. Finally it can show line level coverage analysis graphically.
Btrax is fully GPL'd open source project. All source code can be downloaded from project homepage. Btrax developers are proposing to integrate Btrax kernel facilities into the mainline kernel.
Satoshi Oshima is a software engineer in Linux Technology Center, System Development Laboratory, Hitachi Ltd. He is a member of SystemTap development comunity and he also contributes network (UDP layer) and ext3 filesystem development.
He majored computer science in Tokyo University of Science. He got a masters degree there. He has been enjoying using and hacking Linux kernel since 1995. He had developed a hypervisor for IA32 in 1998-2002. And he has worked for Linux since 2002. He had worked at Red Hat Boston office as an on-site engineer from 2005 to 2007. Currently, he leads Linux Kernel and OSS Virtual Machine Development Team in Linux Technology Center.