Aura

As my colleague Víctor at Igalia has said before in his post, Aura was released to the Nokia Store. Miguel, Víctor and I are quite happy with the result achieved with this app, which intention was to be kind of a port of the Cheese application of the GNOME platform to be used in the N9 or N950 Nokia phones.

The apps allows you to use both cameras (front and principal) to record videos, applying a lot of funny effects (a subset of the GNOME Video Effects) and changing them during the recording. Being Nokia a Finnish company, we decided to name the app after a Finnish Cheese to both honor the GNOME Cheese application and Finland 😉

You can download the app from the Nokia Store where we already got more than 6000 downloads and 100 reviews with a quite good average rating.

You have an example recorded by me with my own phone using the Historical effect and uploaded to Youtube:

And you have even already other videos uploaded to Youtube talking about how Aura works. This one is from a brazilian guy (obrigado!) for FaixaMobi and shows more effects:

Of course, being it free sofware you can also compile it yourself with the code at GitHub and do not be afraid of contributing! The technologies we used were the camerabin element of GStreamer and Qt/QML for the interface where we have the following components:

Aura components UML diagram

  • Main view (aura.qml) with the main interface
  • Controller, which is a mixed QML/C++ object allowing to control the pipeline.
  • Pipeline is a C++ object used by the controller to encapsulate the GStreamer code.
  • PostCapture is also a mixed QML/C++ object that opens the gallery application to show the recorded video and gives you the oportunity of sharing it, deleting it and so on. It uses a C++ controller loaded as a singleton to the context to do some stuff that can only be done in C++. Of course, you can open Gallery yourself and the videos will show up there.
  • EffectManager is a C++ class to load and manage the Effects, which is another C++ class defining how the effect must be applied.
  • Effects (Effects.qml) is a QML component to show the different effects, both software and hardware that Aura can apply. It uses the EffectManager (through the context) to load them and the Controller to apply them.
  • About view (AboutView.qml) is a rework of something done by my colleage Simón Pena and adapted to be used in Aura (Kudos!). It also uses a small AboutViewController to open a Nokia Store URL with the application instead of the browser.
  • ResourceManager is a C++ class used by the Controller to request the proper permissions to record the video.

Invoking Meego 1.2 Harmattan Gallery

As part of my work at Igalia I am writing an app to record some videos for the Nokia N9. I wanted to get them shown in the gallery so I tried the DBUS approach, that is something similar to what is explained here. The problem is the same they faced, meaning that I got gallery in foreground when it was not previously running, but not otherwise.

The solution to that was using libcontentaction. In order to get this working, first you need to get the vendor name in the tags of your pictures or videos, otherwise Tracker will not index them correctly and this solution can be useless.

With the following solution Gallery will be brought to foreground and show the desired file. Code would be something like:

using ContentAction::Action;

[...]

{
    Action action =
        Action::defaultActionForFile(uri,
            "x-maemo-nepomuk/device-captured-video");
    if (action.isValid()) {
        qDebug() < < Q_FUNC_INFO << "chosen action:" << action.name();
        action.triggerAndWait();
    } else {
        qWarning() << "could not file action for" << m_file;
    }
}

The mime type is one of the defined in the galleryserviceaction.desktop that you can find in the device. For images, you can also check the mime types in that file and maybe you do not need to specify it, but I have not tried this and let it to you. Please, comment me your findings.

Mixed QML/C++ objects reloaded

Some days ago I was writing about how to have mixed QML/C++ objects in a QML application for Igalia, but I faced a problem with the code that I had written. And the problem was that I needed to receive some events and from the QML elements I was loading. Retaking that code:

#define QML_PATH "/path/to/the/qml/files/"

MyObject::MyObject(QDeclarativeItem *parent = 0) :
    QDeclarativeItem(parent)
{
    QDeclarativeEngine engine;
    QDeclarativeComponent component(&engine,
        QUrl::fromLocalFile(QML_PATH "MyObject.qml"));
    QDeclarativeItem *rect =
        dynamic_cast(component.create());
    rect->setParentItem(this);
}

and being MyObject.qml something like:

Button {
    id: "myButton"
    text: "Do something cool"
}

The natural way of connecting that button to some slot you are writing would be something like:

MyObject::MyObject(QDeclarativeItem *parent = 0) :
    QDeclarativeItem(parent)
{
    QDeclarativeEngine engine;
    QDeclarativeComponent component(&engine,
        QUrl::fromLocalFile(QML_PATH "MyObject.qml"));
    QDeclarativeItem *item =
        dynamic_cast(component.create());
    item->setParentItem(this);
    connect(item, SIGNAL(clicked()), this, SLOT(doCoolStuff()));
}

Even if you do this, you are not getting any event in the QML button you declare. It behaves as it were a label or whatever, but the key is the engine. The engine must live while you need the events of the QML component. And the easiest way of getting that done is adding the engine as a private class attribute:

#ifndef MYOBJECT_H
#define MYOBJECT_H

#include 
#include 
#include 

class PostCapture : public QDeclarativeItem
{
    Q_OBJECT

// ...

 private slots:
    void doCoolStuff();

 private:
    QDeclarativeEngine m_engine;
};
#endif

and of course removing the engine stack declaration from the constructor:

MyObject::MyObject(QDeclarativeItem *parent = 0) :
    QDeclarativeItem(parent)
{
    QDeclarativeComponent component(&m_engine,
        QUrl::fromLocalFile(QML_PATH "MyObject.qml"));
    QDeclarativeItem *item =
        dynamic_cast(component.create());
    item->setParentItem(this);
    connect(item, SIGNAL(clicked()), this, SLOT(doCoolStuff()));
}

Voilà.

Of course, if you need to connect to any element that is not the root element, you can always forward the signals and properties from the QML root object or just use the QObject::findChild method to access the right component.

Mixed QML/C++ objects

One of the good things of QML is that you can have both C++ and QML code and interact between them. For example, from C++, you can access the QML tree and invoke methods, change properties, etc.

The other way around, you can also define C++ objects and interact with them from QML. Here you have two ways of doing it:

  • Instantiating from C++ and adding the objects to the QML context, meaning that you can invoke the public slots and access the properties.
  • Registering the QML type with qmlRegisterType and then instantiating it from QML.

At the Qt documentation you can find examples about how to implement the different approaches so I won’t talk too much about them and I’ll focus in a special case of the last approach.

Let’s see a QML code like this:

Item {
    id: myObject
    function doCoolStuff() {
        // doing really cool stuff here
    }
    Rectangle {
        anchors.fill: parent
        color: "red"
    }
}

Button {
    text: "Do cool stuff!"
    onClicked: myObject.doCoolStuff()
}

Imagine now that painting that rectangle is something that must be really done by your object, because it is needed and inherent to it, and so is the function. If the code is written only in QML, the answer is obvious, just move the whole Item code to a .qml file and leave the main code like this:

MyObject {
    id: myObject
}

Button {
    text: "Do cool stuff!"
    onClicked: myObject.doCoolStuff()
}

Let’s suppose that you need some C++ code in that object for whatever reason (you want to use some GNOME library, for instance). In this case you need to write it in C++ to define the public slots in that language. Our first step would be something like this:

#include <QtCore>
#include <QtDeclarative>

 class MyObject : public QDeclarativeItem
 {
     Q_OBJECT

// Define some Q_PROPERTY here with its methods

 public slots:
     void doCoolStuff(void);

// We could even define some signals
 };

For the slot, it does not matter if you declare it as slot or with Q_INVOKABLE because QML will see it both ways. Of course, don’t forget to write the cpp file with the implementation for the slot 😉 . And you also need to declare the QML type, for example in your main.cpp file:

#include <QtDeclarative>
#include "myobject.h"

int main(int argc, char *argv[])
{
    // ...
    qmlRegisterType("myapp", 1, 0, "myobject");
    //...
}

So far it is not different from what you can read at Defining new QML elements section of the Qt doc.

But what about painting the rectangle? We can do it with Qt code, but as we are lazy bastards and want to use QML for that, we have two options. One of them is forgetting about it in the C++ code and having something ugly like:

import myapp.myobject 1.0

MyObject {
    Rectangle {
        anchors.fill: parent
        color: "red"
    }
}

But come on, that rectangle is part of the object and nobody should write the Rectangle themselves. So we can take the better approach of moving the Rectangle to a different MyObject.qml file and loading it in the constructor of the C++ class:

#define QML_PATH "/path/to/the/qml/files/"

MyObject::MyObject(QDeclarativeItem *parent = 0) : QDeclarativeItem(parent)
{
    QDeclarativeEngine engine;
    QDeclarativeComponent component(&engine, QUrl::fromLocalFile(QML_PATH "MyObject.qml"));
    QDeclarativeItem *rect = dynamic_cast<QDeclarativeItem *>(component.create());
    rect->setParentItem(this);
}

Voilà.

Being taken to Azkaban for use of very dark GStreamer magic

I was writing some tests for a project at Igalia and I need to mock the convert-frame playbin2 element action. The code to invoke it is something like this:

GstElement *pipeline = /* get pipeline */;
GstCaps *caps = /* create caps to adapt the conversion */;
GstBuffer *buffer = NULL;
g_signal_emit_by_name (pipeline, "convert-frame", caps, &buffer);

When you are writing tests, what you want to do is testing just your code and not to depend on something external, so in this case the idea would be providing a fake implementation for that GStreamer element action.

The way you can do this kind of things is providing the symbol in your code so that the linker when doing its job does not look any further and uses that instead of the one in the external library, so the natural solution coming to your mind would be rewriting g_signal_emit_by_name. The problem with this is that though you are not using it in your code, it is too general, so it is not a good idea.

I thought I could replace the convert-frame action in the playbin2 class, so I wrote this code:

typedef struct
{
  GstPipelineClass parent_class;
  void (*about_to_finish) (gpointer playbin);
  void (*video_changed) (gpointer playbin);
  void (*audio_changed) (gpointer playbin);
  void (*text_changed) (gpointer playbin);
  void (*video_tags_changed) (gpointer playbin, gint stream);
  void (*audio_tags_changed) (gpointer playbin, gint stream);
  void (*text_tags_changed) (gpointer playbin, gint stream);
  GstTagList *(*get_video_tags) (gpointer playbin, gint stream);
  GstTagList *(*get_audio_tags) (gpointer playbin, gint stream);
  GstTagList *(*get_text_tags) (gpointer playbin, gint stream);
  GstBuffer *(*convert_frame) (gpointer playbin, GstCaps * caps);
  GstPad *(*get_video_pad) (gpointer playbin, gint stream);
  GstPad *(*get_audio_pad) (gpointer playbin, gint stream);
  GstPad *(*get_text_pad) (gpointer playbin, gint stream);
} GstPlayBinClass;

static gpointer
gst_play_bin_convert_frame (G_GNUC_UNUSED gpointer playbin,
                            G_GNUC_UNUSED gpointer caps)
{
    GstBuffer *buffer;

    /* Create my own GstBuffer with the data I need */

    return buffer;
}

void
simulator_gst_reset(GstElement *new_pipeline, GstBus *new_bus)
{
    /* ... */

    GstPlayBinClass *klass =
        G_TYPE_INSTANCE_GET_CLASS(new_pipeline, GST_PLAY_BIN_TYPE,
                                  GstPlayBinClass);
    klass->convert_frame = (gpointer) gst_play_bin_convert_frame;

    /* ... */
}

First I declared the GstPlayBinClass copying it from the GStreamer code. I didn’t change any parameters order, just replaced some pointers with gpointer as we don’t need them. This way you don’t break the ABI. Then you can declare your own element action code and finally you get the Class, assign the method and voilà!.

As I said, the solution is far from being the best, but if you know a better way, drop me a comment.

Moving to the next page in MafwGriloSource

Thank Aldon Hynes, who sent me an email with some comments about MafwGriloSource behavior in the N900, I could fix a bug and implement a feature. The bug was a limitation when browsing, as you could only see the first 64 results. It was caused by a problem with indexes when returning the results, as I alwayes sent 0. When fixing this, the interface was requesting more and adding them to the treeview. Then we had all results in the treeview.

Then the problem I saw was that loading so many items in treeview was not slow, but a never ending story, as it took a long time to stop because we were asking for more pages until we reached the end, which took a long time.

I thought of a way of providing my own pagination and the first step was reverting the fix for the index bug to avoid the interface requesting more pages itself, which is a bit hacky, but it was the only way when the interface is not open. The next step was implementing a way of showing a new container row with a “More results…” label to be able to transparently carry on browsing.

Other dirty thing is that I need to override the count and skip parameters because now we have to pay attention to the pagination info and having only 64 results would be a pain in the ass. Do you imaging browsing the thousands and thousands of Jamendo artists in chunks of 64? I thought that 1024 was a much more reasonable number. Anyway a g_message is printed when the given count is overridden.

I talked to Juan about the possibility of implementing that in Grilo itself as it could be a nice of way of providing automatic pagination, but we agreed that it was better to implement the prototype at MafwGriloSource and then, if it was good enough and the model was suitable for Grilo, we could move it there.

How could I implement this? The easiest way was changing the way the MAFW object ids are build from the GrlMedia object. What we had so far, was source_uuid::media_object_id. First I thought of having a special uuid indicating that there was pagination but this would not work as MAFW really needs to parse a valid source uuid (this is, corresponding to an existing source), to send the requests to the appropriate one, so I had to discard this option.

Then, if I could not touch the source uuid, the next and only choice was the media object id. As the MAFW media object id is created directly from the GrlMedia id, it is an opaque string for us, so the best solution was adding the pagination information (the next element index in the list to request) and then catenating the media object id with a semicolon as separation. Result: source_uuid::next_element_index:current_container_media_object_id. For this, I needed to change the functions to serialize and deserialize the object id.

Once I had the pagination info, the only thing left should be changing the callback to return the data to check if there were results left and adding the row that you will see if you, for example, browse Jamendo artists.

Here you have the result:

Screenshot showing pagination

If you find anything weird, drop me a line.

I have to thank Igalia for letting use work time to finish this.