Adapting JHBuild to continuous integration
These days I’ve been doing some work in JHBuild to make life easier for those who want to run it from a Continuous Integration loop. In particular, it should improve the experience for those using Buildbot (as we intend in the Gnome Build Brigade).
Checkout modes for JHBuild #
In Buildbot, most used build scripts enable you to decide how checkouts are done. The options are currently four:
- Update mode: checkout in a directory, then updates over the same folder you’re compiling the tree. It’s the way jhbuild works currently. Problem: sometimes there are generated files in CVS/SVN/whatever. When you update them, they can raise merge conflicts. It’s very typical in gtk-doc templates.
- Clobber mode: it wipes the build directory before checking out. Then, every time you compile, you need to download the full tree. It avoids conflicts as there are no merges. If you use clobber always, it’s good to have a compilation cache to avoid compiling all the trees every time.
- Export mode: similar to clobber, but it uses export instead of checkout. Basically, the difference is you don’t get version control information in the build tree (and then you can use it directly to generate source package, for example).
- Copy mode: works as in update mode, but version control is done in a different directory. Then, after checking out/updating, it copies all the files to the directory it will use to compile. It avoids the merge conflicts problem, enables you to do development over the version control directory being able to create patches easily. Unfortunately it stores the files in version control 2 times.
For me, the most interesting options are the copy mode and the clobber mode. In continuous integration loops, we usually don’t add modifications over the compilation trees, but merge conflicts raise frequently. With those options, you get an easy solution.
I’ve added a bug and a patch to JHBuild bugzilla to support checkout options. With this patch, you can establish a global checkout mode, and specific checkout modes for each module in jhbuildrc file.
Running make check from jhbuild without breaking all the moduleset build every time #
Currently JHBuild lets you run make check for modules using the autotools module type. If you set the makecheck variable in jhbuildrc, it will run make check before installing the packages. The problem with this is: if the checks fail, then JHBuild considers the module is broken, and then it will not compile the modules depending on it.
I’ve written a patch to change this behavior. With this, you can set up a unittestfails variable (currently defaulting to True to maintain current behavior) in jhbuildrc. If this variable is false, make check failures don’t break compilations of the module. It goes on compiling, but shows a message in the log warning about this.
There was an old bug talking about this in JHBuild bugzilla. I added the patch to this bug.
Future work #
Two ideas for JHBuild integration with other tools:
- Convert JHBuild to a Python module. Doing this we could have a version control independent library for python scripts. Buildbot is implemented in Python, so it can be a solution to integrate JHBuild with more flexibility.
- Add DBus support. It would be interesting that applications could fire compilations, and register to wait for different events of the compilations (end of each module, stage or so). Then we could have a better control of the compilation process from external tools.
I’ll be working on this next week.
- Previous: Guadec 2006 experience. Build brigade!
- Next: DBus support for JHBuild