From source code to ndk-build using autotools and androgenizer

This post explains how to compile C source code for Android using the Native Development Kit (NDK) by using autotools to set up the building infrastructure and using androgenizer to convert that autotools infrastructure into Android.mk files understood by the ndk-build tool. I’ll first review some autotools concepts very quickly trough examples and iterate over them.

Simple program

(Download the testapp.tgz example or browse it online)

We start from a simple helloworld program with its main() function that depends on a sayhello() function. We want to compile it using autotools in the simplest possible way.

To create an autotools template from scratch, follow the instructions of this presentation, which are abridged here:

  autoscan
  mv configure.scan configure.ac
  # Add this line below the AC_INIT line:
  # AM_INIT_AUTOMAKE
  nano configure.ac
  autoheader
  aclocal
  touch NEWS README AUTHORS ChangeLog
  automake --add-missing --copy
  autoconf

Makefile.am should contain:

  bin_PROGRAMS = testapp
  testapp_SOURCES = testapp.c

Then, to compile, just:

  ./configure
  make

The testapp.tgz file is an example of this. Look at the “autoall” script.

Library + program using libtool

(Download the testlib.tgz example or browse it online)

In this case we have to add a new libtool line to configure.ac, apart from the automake one, under the AC_INIT line:

  AM_INIT_AUTOMAKE
  AC_PROG_LIBTOOL

This is how Makefile.am should look like:

  lib_LTLIBRARIES = libtestlib.la
  libtestlib_la_HEADERS = testlib.h
  libtestlib_la_SOURCES = testlib.c
  libtestlib_ladir = $(includedir)
  libtestlib_la_LDFLAGS = -avoid-version

  bin_PROGRAMS = testapp
  testapp_SOURCES = testapp.c
  testapp_LDADD = .libs/libtestlib.so
  testappdir = $(includedir)

The LDFLAGS are the flags passed to libtool, and are documented here and here in the Autobook reference. In this case, the “-avoid-version” forces generation of libtestlib.so instead of libtestlib.so.0.0.0 (versioned libs aren’t supported in Android).

UPDATE: This “-avoid-version” flag isn’t needed anymore in recent versions of the NDK. They will generate unversioned libraries automatically and won’t recognize the flag (will show an error).

Look at the “autoall” script in the example for all the details. Now it’s fully automatic, so you don’t have to do the insertions by hand in configure.ac.

Androgenized lib + program

(Download the androgenized-testapp.tgz example or browse it online)

All the C code must reside in a directory called “jni”, inside the main directory of the Android project we want to create (in the androgenized-testapp example only the jni is included, technically the other ones aren’t needed):

androgenized-testapp
  src (Java source code)
  libs (compiled binary libs and executables)
  obj (intermediate object code for libs and exes)
  jni
    Application.mk (actually not mandatory)
    Android.mk (generated by androgenizer, drives the compilation when using NDK)
    Makefile.am
    testlib.c
    ...

A new target called Android.mk must be created in Makefile.am:

Android.mk: Makefile.am
        androgenizer -:PROJECT testlib 

        -:REL_TOP $(top_srcdir) -:ABS_TOP $(abs_top_srcdir) 
        -:SHARED testlib 
        -:SOURCES $(libtestlib_la_SOURCES) 
        -:LDFLAGS $(libtestlib_la_LDFLAGS) 

        -:PROJECT testapp 
        -:REL_TOP $(top_srcdir) -:ABS_TOP $(abs_top_srcdir) 
        -:EXECUTABLE testapp 
        -:LDFLAGS -ltestlib 
        -:SOURCES $(testapp_SOURCES) 
> $@

Androgenizer must be downloaded and installed somewhere in our PATH from here. It has a very brief parameter documentation, and the best way to understand it is by seeing usage examples, such as all the androgenizer lines added to the Makefile.am in the GStreamer project.

In the Android.mk target shown above two modules are going to be compiled: “testlib” and “testapp”. Each module is declared with a “-:PROJECT” parameter. The rest of the lines belong to the current project (current module) until a new one is declared. I’ve left a separation between projects to illustrate this.

Next line should always be a “-:REL_TOP” and “-:ABS_TOP”, as recommended in README.txt. These two values are used to check all the paths in LDFLAGS, CFLAGS, etc. and substitute local paths with absolute paths. This is important, because normal makefiles usually assume that the rules are executed from the subdirectory the makefile is in, while what is considered as the working directory for Android.mk makefiles is always the directory from which ndk-build is invoked. As you can imagine, compiler directives such as “-I..” are going to have a very different meaning and cause the wrong behaviour when passed to the compiler by ndk-build.

After that, we indicate the type of target for the project (“-:STATIC” for libtestlib.a, “-:SHARED” for libtestlib.so, and “-:EXECUTABLE” for testapp). The extension of the library/app is automatically calculated.

The “-:SOURCES” parameter is used to indicate the files to be compiled. In this case we can use the list of files already calculated by the previous automake rules (see testlib.tgz and testapp.tgz examples) and stored in the “libtestlib_la_SOURCES” variable.

The “-:LDFLAGS” parameter is used to generate the needed library dependencies for the current module. In the case of testlib, those dependencies are already calculated by automake in the “libtestlib_la_LDFLAGS”. In the case of testapp, it depends on testlib, so we indicate it manually with “-ltestlib”. That will translate into this line in the generated Android.mk file:

  LOCAL_SHARED_LIBRARIES:=libtestlib

this will make the NDK build scripts to link against the proper “libtestlib.so” (if the lib was dynamic) or “libtestlib.a” (if it was static). The NDK already knows how testlib was built and how it has to be linked.

The final “> $@” line just means that make has to take the output of the androgenizer command and dump it to a file named just like the target it’s being build, that is, “Android.mk”.

After all the autoscan, configure.ac customization, autoheader, aclocal, libtoolize, automake and autoconf steps we can “./configure” and “make Android.mk” to generate the “Android.mk” file. Note that no full “make” is needed. Then we step back one directory (“cd ..” to the project root) and perform the actual build using NDK (which has to be downloaded from here, installed and be in our PATH; the version I used to write this post was NDK r8):

  ndk-build V=1

The V=1 option just prints out all the executed commands, which is very handy. The compilation result will be placed in the “libs” directory.

To understand the process a bit more, it’s a good idea to look into the generated Android.mk file:

  LOCAL_PATH:=$(call my-dir)
  include $(CLEAR_VARS)

  LOCAL_MODULE:=testlib
  LOCAL_SRC_FILES := 
          testlib.c
  LOCAL_LDFLAGS:=
          -avoid-version
  LOCAL_PRELINK_MODULE := false
  include $(BUILD_SHARED_LIBRARY)
  include $(CLEAR_VARS)

  LOCAL_MODULE:=testapp
  LOCAL_SRC_FILES := 
          testapp.c
  LOCAL_SHARED_LIBRARIES:=
          libtestlib
  LOCAL_PRELINK_MODULE := false
  include $(BUILD_EXECUTABLE)

Apart from some boilerplate (LOCAL_PATH, CLEAR_VARS, LOCAL_PRELINK_MODULE), we can distinguish the two modules (LOCAL_MODULE), its source files (LOCAL_SRC_FILES), inherited LDFLAGS (LOCAL_LDFLAGS), required libs (LOCAL_SHARED_LIBRARIES) and finally, the building instruction (BUILD_SHARED_LIBRARY, BUILD_EXECUTABLE).

Seeing the Application.mk and Android.mk supported directives are documented somewhere in your installation of the NDK. They support more options than the ones explained here. With this basic introductions, that documentation should be more understandable. Application.mk can be edited directly. Android.mk has to be customized using the “-:PASSTHROUGH” androgenizer parameters in Makefile.am. For example:

  -:PASSTHROUGH LOCAL_ARM_MODE:=arm

Unfortunately, real life autotools build scripts are a bit more complex than the examples shown here. Here are some debug tips that can be helpful:

  • Use “ndk-build V=1″ to get the exact command line used to compile each file. When something fails, execute that command line independently and examine all the flags passed to it trying to look for incorrect compiler flags or missing include paths. Try to figure out what the correct command line flags would be.
  • Locate where the offending flags come from in the Android.mk. From there, locate them in the Androgenizer Makefile.am target. Usually some Makefile.am variable has the wrong values. Need to debug more on how Androgenizer is called? Go to the desired directory, delete Android.mk, make Android.mk and see the executed command line.
  • Where the Makefile.am variables comes from? What Makefile.am variables should be used? Examine the generated Makefile to find the values of all possible Makefile.am variables. That’s the place to look to decide what to pass to Androgenizer and what not.

14 Responses to “From source code to ndk-build using autotools and androgenizer”

  1. Alain Kalker Says:

    Thanks for the very useful writeup!

    Just a few small problems. First, the download links for the examples don’t work (404s). Also, in the androgenized-testapp, host build (using plain ‘make’) produces a testapp that doesn’t run because the linker cannot find libtestlib.so.
    May I suggest this patch:

    — androgenized-testapp.orig/jni/Makefile.am 2012-01-19 18:50:54.000000000 +0100
    +++ androgenized-testapp/jni/Makefile.am 2012-02-19 20:15:08.453338181 +0100
    @@ -10,7 +10,7 @@ libtestlib_la_LDFLAGS = -avoid-version

    bin_PROGRAMS = testapp
    testapp_SOURCES = testapp.c
    -testapp_LDADD = .libs/libtestlib.so
    +testapp_LDADD = libtestlib.la
    testappdir = $(includedir)

    Android.mk: Makefile.am

    That way, libtool will do it’s magic. :-)

  2. Steve Says:

    Nice post, and possibly the only current documentation for androgenizer ;) Thanks for the write-up.

  3. eocanha Says:

    Alain, I’ve corrected the links issue, thanks for pointing it out. Regarding the problems with dynamic linking, thanks for having risen them in your comment. I’ll explain them a little bit:

    The truth is that dynamic libraries have to be installed in their proper directories for them to be found by the system linker. Those directories are /lib, /usr/lib and those declared in /etc/ld.so.conf and /etc/ld.so.conf.d/*. If the generated libtestlib.so remains in the .libs directory, it’s not going to be found by the dynamic linker on runtime.

    There are some solutions for that:

    Add the .libs directory to the LD_LIBRARY_PATH environment variable (it’s like PATH but for shared libs instead of executables)
    Configure with “./configure –prefix=/”, then “make”, then “sudo make install”, so that the libraries end up installed in the right place. It’s a bad idea to make install if you have a distro based in packages. The right way is to create a package for your distro and install it, but that’s out of the scope of this post. Anyway, you can always do “su make uninstall”.
    Compile statically, as you suggest in your patch. This is always an option, but the interesting goal of the post is actually build a dynamic library, specially for Android

    How to know what libraries our executable depends on, and if they are found by the dynamic linker or not? Just do “ldd testapp” and you’ll see.

  4. Sean Says:

    Anyway to generate recursive Android.mk? I have a project which includes recursively subdirs, there is Makefile.am in each dir. It is burden if I add targets into every Makefile.am.

  5. admin Says:

    Currently I don’t know of any such way. Having one Android.mk per Makefile.am is the solution adopted in GStreamer at the time I wrote the post, though it seems that the SUBDIR parameter helps in some way to do a recursive processing:

    http://cgit.freedesktop.org/gstreamer/gstreamer/diff/Android.mk?id=3f1ef2255a2e7c316d7c877d3a3f6bb815bc79a4

    http://cgit.freedesktop.org/gstreamer/gstreamer/diff/gst/Makefile.am?id=3f1ef2255a2e7c316d7c877d3a3f6bb815bc79a4

    http://cgit.freedesktop.org/gstreamer/gstreamer/diff/gst/parse/Makefile.am?id=3f1ef2255a2e7c316d7c877d3a3f6bb815bc79a4

    However, take into account that the “place” who knows the proper values (sources, ldflags, etc.) needed to compile are exactly each respective Makefile.am in each of the subdirectories. That’s why the Android.mk targets have to be added individually in each of them and then included via a global Android.mk or via the SUBDIR parameter when calling Androgenizer.

  6. theunderdog Says:

    Nice post! Would you be knowing the significance of Android.cfg and how is it used.

    Thanks.

  7. lostintranslation Says:

    >Androgenizer must be downloaded and installed

    Could you please explain this in further details?
    When cloning the Androgenizer i don’t get the configure file for an instance.
    However the configure file is mentioned in the installation instructions here:
    https://github.com/qdii/androgenizer/blob/master/INSTALL

  8. admin Says:

    Androgenizer is so simple that doesn’t need configure. Just:

    $ git clone git://cgit.collabora.com/git/user/derek/androgenizer.git
    $ make

    and, optionally, copy it to somewhere in your PATH. I my case:

    $ cp androgenizer $HOME/bin/

    That’s it!

  9. lostintranslation Says:

    Thanks i did.

    But i got confused by the following message:
    main.c:121:6: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
    int err = 0;
    ^

    I “fixed” the warning by returning err in main instead of 0.
    Hope that is ok?

    Thanks for the quick response :)

  10. admin Says:

    It’s just a warning. Ignore it.

    I have nothing to do with Androgenizer source code. Anyway, your fix looks good to me.

  11. lostintranslation Says:

    I read your comment about setting the LD_LIBRARY_PATH:
    http://blogs.igalia.com/eocanha/2012/01/30/from-source-code-to-ndk-build-using-autotools-and-androgenizer/#comment-216

    I followed the configure, make and make install instructions
    But i still get the following error

    arm-linux-androideabi-g++: error: unrecognized option ‘-avoid-version’
    make: *** [obj/local/armeabi/libtestlib.so] Error 1

    PS. i use gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu9)
    and GNU Make 3.81

    Perhaps my LD_LIBRARY_PATH is incorect.
    Could you please elaborate?

  12. lostintranslation Says:

    Could the ‘-avoid-version’ error have something to do with the recent andrognizer change:
    http://cgit.collabora.com/git/android/androgenizer.git/commit/?id=199eb7b752a18c6e8bd2bdb01216856c3628dd53

  13. admin Says:

    The examples in this post were developed and tested with Android NDK r8, whose compiler *did* support the “-avoid-version” flag. The compiler included in the latest Android NDK r9b (and possibly other previus versions) doesn’t support the “-avoid-version” flag, probably because it generates unversioned libraries in all cases.

    If you want to compile the examples using a recent NDK version and the *old* Androgenizer version I used in the post (git clone http://cgit.collabora.com/git/user/derek/androgenizer.git), you must comment out the line which adds the flag in Makefile.am:

    # LDFLAGS documented here:
    # http://sourceware.org/autobook/autobook/autobook_88.html
    # http://sourceware.org/autobook/autobook/autobook_172.html#SEC172
    # COMMENT OUT THIS LINE:
    # libtestlib_la_LDFLAGS = -avoid-version

    After this change, ./autoall will properly generate the library.

    The commit in Androigenizer that you mention in your comment should delete automatically the “-avoid-version” flag. It’s included in the official repository (git clone git://git.collabora.co.uk/git/android/androgenizer.git, it’s not the same as the one above). If you use Androigenizer coming from that repository, the problem also disappears.

    I’ve updated my post accordingly to point to the right repository. Thank you for forcing me to be aware of this. :-)

  14. lostintranslation Says:

    >After this change, ./autoall will properly generate the library.

    I came to the same conclusion, but wasn’t really sure.
    Had to wonder about it for a couple of hours, before moving on and accepting the nature of things :)
    Thanks for clarifying.