Once Build brigade puts in production the Buildbots for projects, with testing support, we’ll get an automated way to run unit tests of all projects in Gnome umbrella. And you, developer, will love to know when your tests fail to get them fixed asap.
But, to do this, you should implement good tests. How to do these tests? A good unit testing framework is Check.
If you need an example: currently, Iago Toral is developing unit tests for gtk+ with it. You can browse his work or read his post to know about them or ask him at #build-brigade. It’s an interesting work, and accepting and extending these unit tests in mainstream gtk+ would be great for gtk+ quality.
Two main goals in your tests:
- The first goal would be adding tests checking known bugs. If you detect a bug and don’t want it to be regressed you can add a test to know if it happens again.
- The second goal is trying to run as many code paths as you can, in order to be sure they behave as expected, and don’t segfault. Code coverage tools helps you to do it. You can view current coverage of these gtk+ tests.
Wouldn’t it be great that you know your new features don’t break other code behaviour before committing? Just run make check and you’ll begin to be more confident about that.
Of course, these notes apply directly only to C projects. But replace Check with your favourite test framework for your language ;).
I’m interested in the code coverage stuff. What tool is this using and how can I learn more about how to use it?
For this I’m using lcov (http://ltp.sourceforge.net/coverage/lcov.php). This tool reads the logs of a code compiled with coverage options (-fprofile-arcs -ftest-coverage). When you run the tests, the logs are generated, and then lcov lets you explore the logs and generate html reports.
In my jhbuild example you can read the configuration parameters.
# sef FLAGS (with coverage support – gcc4)
#os.environ[‘CFLAGS’] = ‘-g3 -O0 -fprofile-arcs -ftest-coverage’
#os.environ[‘CXXFLAGS’] = ‘-g3 -O0 -fprofile-arcs -ftest-coverage’
#os.environ[‘LDFLAGS’] = ‘-lgcov’
# sef FLAGS (with coverage support – gcc3)
#os.environ[‘CFLAGS’] = ‘-g3 -O0 -fprofile-arcs -ftest-coverage’
#os.environ[‘CXXFLAGS’] = ‘-g3 -O0 -fprofile-arcs -ftest-coverage’
The link to the script using these logs in bonsai:
http://bonsai.fisterra.org/cgi-bin/bonsai/cvsblame.cgi?file=jhbuild-buildbot-scripts/slave/scripts/module-reports.sh&rev=&root=/var/publiccvs
And the link to the script that runs the tests (and cleans the previous coverage logs).
http://bonsai.fisterra.org/cgi-bin/bonsai/cvsblame.cgi?file=jhbuild-buildbot-scripts/slave/scripts/make-check.sh&rev=&root=/var/publiccvs