Testing C with Libtap

Libtap is a library for testing C code. It implements the Test Anything Protocol, which is emerging from Perl's established test framework.

Design for Today, Code for Tomorrow

One of the ideas behind Extreme Programming (XP) is to "design for today, code for tomorrow." Rather than making your design cover all eventualities, you should write code that is simple to change should it become necessary.

Having a good regression test suite is a key part of this strategy. It lets you make modifications that change large parts of the internals with the confidence that you have not broken your API. A good test suite can also be a way to document how you intend people to use your software.

Having worked where people thought that writing tests was a waste of time, I can't tell you how much time I wasted trying to fix bugs that had emerged as a result of bugs being fixed or new features added. If we'd had a proper regression test suite, we could have found those immediately, and I would have lots of extra time to write new features. Taking the time to produce good tests (and actually running them) actually ends up saving a lot of time, not wasting it.

Introducing the Test Anything Protocol

Perl distributions normally ship with a test suite written using Test::Simple, Test::More, or the older (and now best avoided) Test module. These modules contain functions to produce plain-text output according to the Test Anything Protocol (TAP) based on the success or failure of the tests. The output from a TAP test program might look something like this:

1..4
ok 1 - the WHAM is overheating
ok 2 - the overheating is detected 
not ok 3 - the WHAM is cooled 
not ok 4 - Eddie is saved by the skin of his teeth

Related Reading C in a Nutshell By Peter Prinz, Tony Crawford Table of Contents Index Sample Chapter Read Online--Safari Search this book on Safari:   Only This Book All of Safari Code Fragments only

The 1..4 line indicates that the file expects to run four tests. This can help you detect a situation where your test script dies before it has run all the intended tests. The remaining lines consist of a test success flag, ok or not ok, and a test number, followed by the test's "name" or short description. Obviously, the second and third lines indicate a successful test, while the last two indicate test failures.

Perl modules usually invoke the tests either by running the prove program or by invoking make test or ./Build test (depending on whether you're using ExtUtils::MakeMaker or Module::Build). All three approaches use the Test::Harness module to analyze the output from TAP tests. If all else fails, you can also run the tests directly and inspect the output manually.

If Test::Harness is given a list of tests programs to run, it will run each one individually and summarize the result. Tests can run in quiet and verbose modes. In the quiet mode, the harness prints only the name of the test script (or scripts) and a result summary. Verbose mode prints the test "name" for each individual test.

Besides Perl, helper libraries for producing TAP output are available for many languages including C, Javascript, and PHP (see the Links & Resources section).

Suppose that you want to write tests for the module Foo, which provides the mul(), mul_str(), and answer() functions. The first two perform multiplication of numbers and strings, while the third provides the answer to life, the universe, and everything. Here is an extremely simple Perl test script for this module: