Boosting WebKitGTK+ compilation for armhf with icecream

Some time ago I needed to jump into the fix-compile-test loop for WebKitGTK+, but in the armhf architecture, speaking in terms of Debian/Ubuntu.

To whom don’t know, WebKitGTK+ is huge, it is humongous, and it takes a lot of resources to compile. For me, at first glance, was impossible to even try to compile it natively in my hardware, which, by the way, is an Odroid-X2. So I setup a cross-compilation environment.

And I failed. I could not cross-compile the master branch of WebKitGTK+ using as root file system, a bootstrapped Debian. It is supposed to be the opposite, but all the multiarch thing made my old and good cross-compilation setup (based on scratchbox2) a bloody hell. Long story short, I gave up and I took more seriously the idea of native builds. Besides, Ubuntu and Debian does full native builds of their distributions for armhf, not to say that the Odroid-X2 has enough power for give it a try.

It is worth to mention that I could not use Yocto/OE or buildroot, though I would love to use them, because the target was a distribution based on Debian Sid/Ubuntu, and I would not afford a chroot environment only for WebKitGTK+.

With a lot of patience I was able to compile, in the Odroid, a minimalist configuration of WebKitGTK+ without symbols. As expected, it took ages (less than 3 hours, if I remember correctly)

Quickly an idea popped out in the office: to use distcc. I grabbed as many board based on ARMv7 I could find: another Odroid-X2, a couple Pandaboards, an Arndaleboard, and an IFC6410, installed in them a distcc compilation setup.

And yes, the compilation time went down, but not that much, though I don’t remember how much.

Many of the colleagues at the office migrated from distcc to icecream. Particularly, Juan A. Suárez told me about his experiments with icecc and his Raspberry pi. I decided to give it a shoot.

Icecream permits to do cross-compilation because the scheduler can deliver, into the compilation host, the required tool-chain by the requester.

First, you should have one or several cross tool-chains, one for each compilation tuple. In this case we will have only one: to compile in X86_64, generating code for armfh. Luckily, embdebian provides it, out of the box. Nevertheless you could use any other mean to obtain it, such as crosstool.

Second, you need the icecc-create-env script to create the tarball that the scheduler will distribute to the compilation host.

    $ /usr/lib/icecc/icecc-create-env \
          --gcc /usr/bin/arm-linux-gnueabihf-gcc-4.7 \
                /usr/bin/arm-linux-gnueabihf-g++-4.7

The output of this script is an archive file containing all the files necessary to setup the compiler environment. The file will have a random unique name like “ddaea39ca1a7c88522b185eca04da2d8.tar.bz2” per default. You will need to rename it to something more expressive.

Third, copy the generated archive file to board where your code will be compiled and linked, in this case WebKitGTK+.

For the purpose of this text, I assume that the board has already installed and configured the icecc daemon. Beside, I use ccache too. Hence my environment variables are more or less like these:

CCACHE_PREFIX=icecc
CCACHE_DIR=/mnt/hd/.ccache # /mnt/hd is a mounted hard disk through USB.
PATH=/usr/lib/ccache:..    # where Debian sets the compiler's symbolic links

Finally, the last pour of magic is the environment variable ICECC_VERSION. This variable needs to have this pattern

<native_archive_file>(,<platform>:<cross_archive_file>=<target>)*.

Where <native_archive_file> is the archive file with the native tool-chain. <platform> is the host hardware architecture. <cross_archive_file> is the archive file with the cross tool-chain. <target> is the target architecture of the cross tool-chain.

In my case, the target is not needed because I’m doing native compilation in armhf. Hence, my ICECC_VERSION environment variable looks like this:

ICECC_VERSION=/mnt/hd/tc/native-compiler.tar.gz,x86_64:/mnt/hd/tc/arm-x86-compiler.tar.gz

And that’s it! Now I’m using the big iron available in the office, reducing the time of a clean compilation in less than an hour.

As a final word, I expect that this compilation time will be reduced a bit more using the new cmake build infrastructure in WebKitGTK+.

How I installed the Linaro’s toolchain

Last week, because of a distraction, I dd’ed my external hard disk, where I used to store my cross compilation setup. And yes, I lost everything: my OpenEmbedded working directory, my installation of the CodeSourcery toolchain, and a lot more.

But, as the people say, when the going gets rough, the rough get going. So I considered this mistake as a sane tabula rasa to start all over again, learning a few new tricks along the process.

My objective was to substitute the CodeSourcery toolchains for the Linaro’s one, because my main development boards are a beagle-board and a panda-board.

But I use Debian, not Ubuntu, where the Linaro’s toolchain can be deployed out-of-the-box. I had to compile it. For that I came aware of crosstool-ng, a nice utility, based on kconfig, to build toolchains, and also it supports Linaro in its last release.

To have only a toolchain is not enough. For example, my syslink wrapper depends upon libelf. A cross-compiled sysroot is also required. And for that I chose buildroot, which is another kconfig based utility to build complete embedded systems. And it integrates with crosstool-ng too!

Sadly, the version of crosstool-ng handled currently by buildroot is a bit older and do not include the Linaro’s toolchain. Nevertheless it was pretty much easy to modify in order to use the latest release of crosstool-ng (patch).

Now I will try to work with this environment, which, at first sight, looks comfortable and neat.

I have to add that OE is quite complete and it also includes the Linaro’s toolchain. Just I wanted to explore other options this time.