MAFW went Grilo

As I explained in another post, some colleagues at Igalia were creating the Grilo framework to gather multimedia content and ease the creation of that kind of applications. The origins of Grilo are in MAFW, which is the multimedia application framework used in Maemo 5 (Fremantle) to power the official media player, and that we had created in collaboration with Nokia and other companies, but we wanted to go beyond its limitations and Grilo is the result.

We thought that we could mix both things and power MAFW with Grilo, by creating a MAFW source so that contents provided by Grilo could be accessed through MAFW and therefore, through the official Fremantle media player. I began to code that and I already got something pretty stable. You can find it at Gitorious.

For grilo and grilo-plugins, I created fremantle branches from the last stable release at that moment. Some plugins became useless because of two main reasons: they had already the counterpart in MAFW (such as UPnP or bookmarks) or they couldn’t be mapped correctly to MAFW, such as Apple trailers (which needed a user-agent to work) or Vimeo, which does not support browse, so I removed them from grilo-plugins fremantle branch release.

Other important thing I had to do was porting some plugins from GVFS back to GnomeVFS because GVFS http backend was not working propertly and it made the plugins hang.

It is a pity that I couldn’t rebase against the last grilo release, because of GData. The fremantle release is not working properly with Grilo, so I preferred to have the last stable release working.

I received some feedback from the Maemo community. They were requesting a couple of things, but none of them is feasible or has a proper solution. First is remembering the position at least for Filesystem. Problem is that who sends the position to be stored is the interface and it only does it for the mafw-tracker-source, so it is something we cannot do if they don’t send that positions to us.

Other interesting feature is the search and that’s a problem. MAFW does not support search (one of MAFW limitations we corrected in Grilo), though it could be easily wrapped as a special browse. The biggest problem is the interface, which we don’t have access to. As the interface does not have any search button and has no possibility to have it, it makes the thing almost impossible. There could be the solution of adding a search container and it that was opened, we could allow selection the search string by tapping on rows with letters, but you’d need to tap and pan many times to perform the search, which makes it annoying and of course, very ugly.

Summing up, no search and no remembering the last position because the interface does not support those things. What I am trying to do now is getting MAFWGriloSource listed in the App Manager. Lucky that I know two fellow Igalians, who hacked on it 🙂 .

Lightning talk for GUADEC submitted

My submission for this year’s GUADEC:

This is a _lightning talk_ about how the evolution from MAFW (Multimedia Aplication FrameWork used in Fremantle official media player) to Grilo (new multimedia framework for application aiming to provide easy access to many sources of media) and how they can work together to provide a better user experience and access to more media in the Maemo 5 platform (Fremantle)

mafw goes Grilo

As you may know, some colleagues at Igalia are developing a framework to gather, browse and query multimedia sources called Grilo. Of course it is no replacement for GStreamer as it is at a much higher level and we are focusing in gathering, browsing and querying so far.

We were an important part of the main developers of MAFW, so in this case we are trying to learn from the mistakes and try to create a more useful and easy to use framework.

One of my first steps, as a test, will be creating a MAFW pluging for Grilo, so that we can have all sources managed by Grilo running on the Fremantle official media player (as soon as bug 9361 gets fixed) with the consequence of having integrated important and interesting features as Youtube, podcasts, Jamendo, Shoutcast and so on.

mafw-gst-renderer and volume management

In the beginning we are just having a volume in MAFW that we were just setting to the GStreamer. This was a bit complicated to maintain because you always have to be careful of resetting volume to the pipeline when you create a new one and even after some state changes, meaning GST_STATE_READY for example, what I consider a GStreamer limitation, BTW. Anyway, we kept that with a conditional compilation because running Pulse was hell inside Scratchbox, at least for me.

Maemo 5 does not have global volume (to be changed by a user) and uses Pulseaudio, as I just advanced. It defines several volume roles so we can say that volume is context sensitive, meaning that call volume is not the same as multimedia volume, for example. As Maemo can have more than one media player it was not a clever idea to define our volume and use it with GStreamer pipeline and if you also take into account that you need to have the same volume as the status menu volume bar, mission turns much more complicated. The idea was then to use Pulse directly through ext-stream-restore2 API (which I think is not merged upstream yet). One of the side effect advantages is that we do not need to store the volume between executions because Pulse does.

Pulse API has the ability of several kind of loops, say, a threaded mainloop and a GLib mainloop. For simplicity and as it was enough, we used GLib mainloop API.

Other issue was merging requests implementation. We implemented this by sending only a maximum of 5 changes per second, it is once every 200ms in order not to flood Pulse with changes.

One of the things I did not like too much was the hacks to reconnect to pulse if they failed. In those cases we were just crashing, but people did not like it, so we had to implement reconnection (IMHO Pulse should never crash either). The issue was failures during reconnection, which made us to write a hack to try reconnection after one second, until we get it. This solved the problems but it is a workaround, not a solution (which should be Pulse not crashing or reconnecting always fine).

You can take a look at the code at mafw-gst-renderer-worker-volume.c and mafw-gst-renderer-worker-volume.h.

Seekability and DLNA in MAFW

Seekability when streaming contents involves almost all layers of a multimedia player and it is not a trivial issue. First, your interface needs to have a seekbar or something to do that. Of course, the media you are streaming has to seekable, meaning container and codecs used. And we cannot forget the transport either, this is, HTTP, local access and so on.

Thank Gods, GStreamer does a wonderful job making easier everything related to transport, decoding, containers, etc. But in the case of UPnPDLNA, there is a extension saying if the media you are playing is seekable or not.

We did not have support for that in MAFW and we needed it, I added the request for the metadata key MAFW_METADATA_KEY_IS_SEEKABLE when requesting data to play media, so now we have duration, uri and seekability. Our approach had to be consevative (otherwise, GStreamer seekability would have been enough), and then if MAFW source provides the metadata key and media is not seekable, se just say it is not seekable. Otherwise, we rely on GStreamer, that has the final responsibility (which is logical, if it cannot seek, seeking is impossible). Of course, seekability depends on duration, meaning, if there is no duration, we cannot seek as we would not know the seeking limits.

About how to implement that in the sources, the only one needing it was mafw-upnp-source. In the other ones we just want to rely on GStreamer, but for UPnP, if we want to honor DLNA specification, we had to do that. Specification is fun itself and of course, adding that every vendor/provider implements what it wants makes it more difficult. A proof is that a friend of mine bought a TV claiming to be DLNA certified and it only works with the crappy server provided by the vendor, closed of course, and with a lot of missing features, like subtitles support, IIRC (yes Zeenix, I told him to use Rygel and Philippe, it was before you joining Igalia :-p). Now with new gupnp this was reworked, but when I wrote it, we had to check the DLNA_OP fields to know if it was seekable or not and it was a pain in the ass to decide what the default was depending on the missing options and so on.

Fortunately, it is working fine now, I think.

State pattern fixed in the mafw-gst-renderer

I had this post planned for a long time, but here it goes.

When we were developing the renderer some time ago, we saw that code was getting out of control because handling state changes was becoming hell, so we decided to rework it applying the State Pattern. Though Zeenix thinks it was my wife‘s idea, it was actually Iago‘s.

The other day I found a function that was sinning against that pattern. It was _update_playcount_cb. When it was written, the call had been place in a central point where every media change going thru, but it was a bit ugly as it was using an if statement checking for the current state. This application of the state pattern is not completely canonical as we keep the current state but only because state classes are stateless and we want to keep them as Singleton’s, so the easiest way was keeping that variable to point the current state, but of course using it in an if statement is ugly.

Fortunately, I checked where it was needed to call that function and it saw it was in couple of places, so I reworked the code a bit and place the calls in the exact places where it was needed, so i didn’t need to make that function support the state pattern (in fact it would have been stupid).

But this state pattern still needs some finetuning. We’d need to remove the worker and increase the number of states and add substates to control for example the buffering and hide properly some GStreamer state changes that would make the code easier to understand and maintain.

Conic in mafw-gst-renderer

Should we rely on GStreamer timeouts when handling disconnections or should we write some code with conic to immediately detect if network goes down and raise the proper error? Good question, uh?

Issues are network changes and how conic handles them. Because AFAIK, if conic changes it connection it signals the disconnection and then the connection to the other network, (for example if you switch from one WLAN to another or it switches from WLAN to Bluetooth GPRS or viceversa). Issue is that we don’t handle connections ourselves but GStreamer does. They rely in underlying components, such souphttpsrc (which happened to have a problem with the timeout) so they raise an error when that timeout expires. They have more components dealing with different kind of servers that can behave in a different way.

When network goes down and other one comes, we would have to ask for other connection if we are currently playing a stream, but I don’t know if those GStreamer underlying components can handle those connection changes gracefully because the underlying protocols can be stafeful or stateless.

In those cases, we would need to handle those changes ourselves. We would need to set a timeout to see if a new connection event comes or raise the error if not and if it comes, hope that GStreamer can handle that change gracefully.

We think those changes would make the code more complex and wouldn’t fix problems perfectly since maybe GStreamer cannot handle it, so we think it is better to keep things as they currently are, raising the error when GStreamer says so and we don’t have a connection.

MAFWing

Yes, finally it is public that we are contributing to MAFW project, born for the Maemo platform.

My latest jobs were focused in the mafw-gst-renderer component so feel free to ask me any questions related to that or any other component.

I am damm lazy to blog, but I will try to do it sometimes to talk about what I did for the project so that you can stay tuned and follow a bit project’s evolution.

Small things

These last days I touched some small issues as changing a bit the way we destroy the GStreamer pipeline, since we we checking for asynchronous state changes when setting it to NULL that should never happen.

Compiling Epiphany+WebKit with jhbuild under debian amd64

The next challenge I faced with my debian amd64 was compiliing Epiphany with WebKit. As my system is ‘almost virgin’ I had to install many libraries deveplopment packages that you won’t need to install if you have played enough with your system.

First step if you have a dual core computer, activate it in .jhbuildrd with the line:

os.environ['MAKE'] = 'make -j2'

I guess it should work if you just define the environment variable in the .bashrc.

The first problem comes when compiling the bootstrap because gmp reminds you that you are under a 64 bits architecture. Configure stops because it can’t find a suitable compiler. I guess this should be fixed in jhbuild or just in gmp configure.in file because it doesn’t recognize the architecture correctly. To bypass this problem it is needed to specify the architecture to the package or for the whole jhbuild system. I preferred to avoid this problem for all packages and just added it in .jhbuild as:

autogenargs='--build x86_64-linux-gnu'

Actually the first problem you find is that gettext and autoconf stop compiling because of a wrong dependency in make process that causes it to try to compile something that wasn’t compiled yet. It is annoying but to bypass it just carry on compiling.

Next problem came with guile. I say ‘came’ instead of ‘comes’ because when doing this again to test the steps problem had dissapeared. Maybe they fixed it, but it couldn’t find the gmp library that he had just compiled before. With external help we realized that and bypassed it specifying the place where libraries are:

module_autogenargs['guile'] = autogenargs + ' --libdir /yourgnomebuildinstallationdirectory/lib64'

Until this moment, we were only compiling the bootstrap. Then you can begin with epiphany with jhbuild build epiphany. Next problem comes with gnome-doc-utils:

I/O error : Attempt to load network entity http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
warning: failed to load external entity "http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl"
error
xsltParseStylesheetFile : cannot parse http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
compilation error: file ./xml2po.1.xml line 17 element refentry
xsltParseStylesheetProcess : document is not a stylesheet

What was missing was the package docbook-xsl. Next problems come with cairo:

configure: error: flex is required to create the libIDL scanner
*** error during stage configure of libIDL: ########## Error running ./autogen.sh --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --build x86_64-linux-gnu ***
configure: error: yacc is not usable as yacc - consider using bison
*** error during stage configure of libIDL: ########## Error running ./autogen.sh --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --build x86_64-linux-gnu ***

Fixed if installing packages flex and bison. Other problem with cairo is libpng. You need to install libpng-dev package but this is not enough because jhbuild replaces PKG_CONFIG_PATH environment variable, which causes it not to find it. Just adding the a line solves the problem (and many others that may appear) because it will search in system’s pkg-config system. This a solution but also a drawback: if any package configuration is not well written, it could cause you to use the system configuration instead of the compiled with jhbuild, though this should be a very rare thing. Line is:

addpath('PKG_CONFIG_PATH', '/usr/lib/pkgconfig')

While compiling GTK, avahi,icon-naming-utils and startup-notification I got:

* configure: WARNING: *** TIFF loader will not be built (TIFF library not found) ***
configure: error:
*** Checks for TIFF loader failed. You can build without it by passing
*** --without-libtiff to configure but some programs using GTK+ may
*** not work properly
*** error during stage configure of gtk+: ########## Error running ./autogen.sh --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --build x86_64-linux-gnu ***
configure: error:
*** Checks for JPEG2000 loader failed. You can build without it by passing
*** --without-libjasper to configure
*** error during stage configure of gtk+: ########## Error running ./autogen.sh --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --build x86_64-linux-gnu ***
*** error during stage configure of avahi: ########## Error running ./configure --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --disable-qt3 --disable-qt4 --disable-mono --disable-monodoc --disable-manpages --disable-python --enable-compat-howl --enable-compat-libdns_sd --build x86_64-linux-gnu ***
checking for XML::Simple... configure: error: XML::Simple perl module is required for icon-naming-utils
*** error during stage configure of icon-naming-utils: ########## Error running ./configure --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --build x86_64-linux-gnu ***
configure: error: X11 development libraries/headers not found
*** error during stage configure of startup-notification: ########## Error running ./configure --prefix /home/calvaris/nocrypt/gnome/install --libdir '${exec_prefix}/lib64' --build x86_64-linux-gnu ***

You need these packages for that:

  • libtiff4-dev
  • libjasper-dev
  • libgdbm-dev
  • libxml-simple-perl
  • libxt-dev

And then comes a very funny thing: gnome-desktop stops because it doesn’t find the foundation-members.list files. If taking a look at the autogen.sh (thanks, garnacho) you see the following:

if test "`which gvfs-copy`" != ""; then
gvfs-copy http://api.gnome.org/gnome-about/foundation-members gnome-about/foundation-members.list
else
touch gnome-about/foundation-members.list
fi

In this case problem is that it finds gvfs-copy (it had been compiled before), but it couldn’t copy the file and you don’t get it eventually. A bug exists already about that but it was close as invalid, so I have to open a new one to upload a little patch I created. To bypass the problem immediately, you only need to type:
touch gnome-about/foundation-members.list

After continuing you can stop it at epiphany’s beginning, because you neither compiled nor installed webkit yet. I’ll carry on explaining how to compile webkit and epiphany in the next post. Enough for today.