OpenMAX: a rant

I used to work with OpenMAX a while ago. I was exploring an approach to wrap OpenMAX components with GStreamer elements. The result was gst-goo. Its purpose was to test the OpenMAX components in complex scenarios and it was only focused for the Texas Instruments’ OpenMAX implementation for the OMAP3 processor.

Some time after we started gst-goo, Felipe Contreras released gst-openmax, which had a more open development, but with a hard set of performance objectives like zero-copy. And also only two implementations were supported at that moment: Bellagio and the TI’s one mentioned before.

Recently, Sebastian Dröge has been working on a redesign of gst-openmax, called gst-omx. He explained the rational behind this new design in his talk in the GStreamer Conference 2011. If you are looking for a good summary of the problems faced when wrapping OpenMAX with GStreamer, because of their semantic impedance mismatch, you should watch his talk.

In my opinion, the key purpose of OpenMAX is to provide a common application interface to a set of different and heterogeneous multimedia components: You could take different implementations, that could offer hardware-accelerated codecs or either any other specialized ones, and build up portable multimedia applications. But this objective has failed utterly: every vendor delivers a incompatible implementation with the others available. One of the causes, as Dröge explained, is because of the specification, it is too ambiguous and open to interpretations.

From my perspective, the problem arises from the need of a library like OpenMAX. It is needed because the implementer wants to hide (or to abstract if you prefer) the control and buffer management of his codification entities. By hiding this, the implementer has the freedom to develop his own stack closely, without any kind of external review.

In order to explain the problem brought by the debauchery in the hind of OpenMAX, let me narrow the scope of the problem: I will not fall on the trap of portability among different operative systems, specially in those of non-Unix. Even more, I will only focus on the ARM architecture of the Linux kernel. Thus, I will not consider the software-based codecs, only the hardware-accelerated ones. The reason upholding these constrains is that, beside the PacketVideo’s OpenCORE, I am not aware of any other successful set of non-Unix / software-based multimedia codecs, interfaced with OpenMAX.

As new and more complex hardware appears, with its own processing units, capable of off-loading the main processor, silicon vendors must deliver also the kernel drivers to operate them. This problem is very recurrent among the ARM vendors, where the seek of added value gives the competitive advantage, and the Linux kernel has the virtues required for a fast time to market.

But these virtues have turned into a ballast: It has been observed excessive churn in the ARM architecture, duplicated code, board-specific data encoded in source files, and conflicts at kernel’s code integration. In other words, every vendor has built up their own software stack without taking care of developing common interfaces among all of them. And this has been particularly true for the hardware-accelerated multimedia components, where OpenMAX promised to the user-space developers what the kernel developers could not achieve.

First we need a clear and unified kernel interface for hardware accelerated multimedia codecs, so the user-space implementations could be straight, lean and clean. Those implementations could be OpenMAX, GStreamer, libav, or whatever we possibly want and need.

But there is hope. Recently there has happened a lot of effort bringing new abstractions and common interfaces for the ARM architecture, so in the future we could expect integrated interfaces for all these new hardware, independently of the vendor.

Though, from my perspective, if we reach this point (and we will), we will have less motivation for a library like OpenMAX, because a high level library, such as GStreamer, it would cover a lot of hardware within a single element. Hence, it is a bit pointless to invest too much in OpenMAX or its wrappers nowadays.

Of course, if you think that I made a mistake along these reasons, I would love to read your comments.

And last but not least, Igalia celebrates its 10th anniversary! Happy igalian is happy :)

Tags: , , ,

3 comments

  1. Completely agree on many points.

    But gst-openmax was started before gst-goo :)

    Also, I find it amazing that Sebastian Dröge is starting yet another wrapper, and doing it silently (to my knowledge) without even consulting me. Also, clearly copying large chunks of gst-openmax without the proper copyright notices, which is illegal.

    Anyway, I liked the idea of OpenMAX, until I realized it has one fatal flaw; it’s not openly governed, so really, there’s not much “open” in OpenMAX. The open source community has no say on what’s going on, like say, v4l2, or FFmpeg, where all the API changes are publicly discussed and everyone has a say, including companies.

    I also agree that all these user-space API’s for hardware codec acceleration are pointless–va-api, vdpau, xvmc, OpenMAX, etc. We need a kernel interface, but in fact, there is already one; v4l2. Samsung already expose their codecs to user-space this way.

    I plan to do the same with the OMAP 3 stuff (although the tidspbridge needs to be cleaned up too). And then, it should be rather straight-forward to write a single GStreamer wrapper for such codecs, and be done with all this user-space APIs nonsense.

    As usual… the problem is lack of time :(

  2. […] is in my opinion a great initiative which has the potential to make the final spec a better one by opening the discussion to a wider community of […]

  3. Although rather “dated” by now, thank you for this RANT”§$%!
    You saved me from summarizing largely similar (but not nearly as coherent) thoughts about the STILL (1,5 years later) !! closed/incompatible/not emulatable/not debuggable/even hardware manufacturing date dependent/not updatable !! implementations of openMAX.
    From my current perspective of competing rather than undestandable and “defining” away company secret code, along with 5 to 12 closed source libraries, I would pronounce “OPEN”max dead…

    … if it wasn’t for openGL based on EGL and “open” HTML 5. I have to “provide the usage” of both of them and you just can’t do this without buying or licensing closed source.
    Especially the google chrome experiments (as well as other WebGL or canvas examples including h24 video playback) will NEVER run at modern interactive framerates (i measure according to stenvens98 with ~FPS>22) without 100.000 [hours*man] (~= 2,5 Million Euro)

    Back to the main topic: A standard that can only be realized/implemented for a specific monetary value is not “open” – because it is only open to those who can afford that sum.
    A standard that, because of it’s closed source dependencies, can only be implemented to run on/support a limited number of hardware/software configurations is not “open” – because it is not limited by it’s implementation but by its “disclosed” dependencies.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>