elima – Page 2 – Bless the uncertainty

The Web jumps into D-Bus

Lets start with the fun. The following demos show the EventDance’s D-Bus bridge in action:

Proxying org.freedesktop.Notifications

Directly from a Web page, we create a proxy to the standard freedesktop.org object /org/freedesktop/Notifications which is exported to the session bus running in my normal desktop environment. Then, we call the remote method Notify using a simple web form. This method opens a notification dialog showing a quick note to the desktop user. The proxy also watches the remote signal NotificationClosed telling that a notification dialog was closed.
During the demo we open different browsers to show the cross-browser nature of EventDance’s Web transport.

Your browser doesn’t seem to support HTML5 video tag but you can download the video directly.

Screencast 1: D-Bus proxy example
Acquiring a bus name and exporting an object

In the next screencast we acquire/release a name on the session bus, and export a simple object onto it. The object has one method Ask that receives a string as input argument, shows a confirmation dialog on the Web page, and return a boolean value depending on whether the user pressed “Ok” or “Cancel” to close the dialog. We use D-Feet to call the remote method on the browser that owns the name at the moment of the call.

Your browser doesn’t seem to support HTML5 video tag but you can download the video directly.

Screencast 2: D-Bus name owning example
The client-side source code of both demos is available at examples/common/dbus-bridge-proxy.html and examples/common/dbus-bridge-own-name.html respectively, inside EventDance repository. You can check them to realize how simple is the API to connect to and use the D-Bus connection.

The server-side code for both demos is the same, and is public at examples/dbus-bridge.c. There is also a JavaScript version at examples/js/dbusBridge.js and a python version at examples/python/dbus-bridge.py. To run JavaScript and python examples you need EventDance compiled with –enable-introspection configure flag active (the default), and GJS or pygobject respectively available in your system.

The D-Bus bridge

At the highest level, EventDance library provides a set of transports and IPC mechanisms. The D-Bus bridge is just one of these mechanisms, written specifically to work over the Web transport. Its purpose is to connect a Web page with a remote D-Bus daemon running on the Web server, using the EventDance’s Web transport (long-polling or websocket if available).

D-Bus uses a binary protocol, and the browser’s JavaScript context doesn’t (yet) support binary data. Thus, it was sensible to translate D-Bus messages to a browser-friendly protocol, in this case JSON. Using the API recently added into json-glib to integrate GVariant and JSON, the bridge translates messages back and forth from GDBus to JSON and vice-versa. Also, it tries to reduce message payload as much as possible, by caching the GDBus objects server-side, and sending only their references over the wire. This makes the protocol very simple and lightweight.

Cool, now what?

Hopefully you are already wondering about the many purposes this feature can serve.

While many people envision that the next desktop will be the browser, many more do use Web applications almost exclusively, already today. The traditional separation between Web and Desktop app development is blurring. Browsers have become powerful platforms for running complex applications, and this situation is speeding up with the broad and increasing adoption of HTML5 standards by major browsers.

On the other hand, we have D-Bus, a freedesktop.org standard that is at the core of almost every GNU/Linux system out there. It is the de-facto IPC mechanism on which your applications talk and share. D-Bus allows us to write a program in any language, and export its usefulness over a standard channel. Also allows us to write differentiated UIs (e.g, Qt vs. GTK+ vs. NCurses) to interface a common functionality. Yes, one bus to bind them all!

Joining these two pieces together is just the next logical step. A step towards bringing together the best of two contexts: the ubiquity of the Web and the inter-process collaborative nature of the Desktop.

We need to write applications that you can host and use securely and reliably not only in your computer, but anywhere in the Planet where you happen to have a browser plugged into the Net; whether it is your laptop, mobile phone, tablet or your neighbors’ PS3. We also need to encourage application developers to export the logic of their programs over D-Bus, to allow other platforms (like the Web!) to reuse it. Telepathy is a good example of such program.

Almost identical applications in terms of functionality are written for many FOSS environments like GNOME, KDE, MeeGo, Android, etc; yet many times only the user experience and the technologies used to build it are different. Although not always possible, there is room for a wider code-reusing culture if we come back to the original Unix philosophy:

Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

That universal interface has just been upgraded.

JSON and GVariant

Last week, my patch to provide GVariant integration into json-glib was merged. I filed a bug for it back in October, and after some patching iterations it finally got its way in.

This mean that now you can obtain a JSON node tree from a GVariant value and the opposite, with a single API call. I want to thank Emmanuele for reviewing my patches promptly (they were kind of lengthy), and for the positive feedback. The new API will be available in next release 0.14.

Why do we need that, anyway?

This integration seems quite natural if you give it a thought:

json-glib and GVariant are both general purpose data structure holders,
with serializing/deserializing capabilities
and both are GLib based.

My motivation to hack on that comes from my work in EventDance. I’m trying to bring D-Bus APIs into a Web page so that it is possible to connect to any server-side message bus from a page’s script and talk to the exported objects, export JavaScript objects to the bus so that they can be consumed by server applications, and own bus names. This feature and the rationale behind its usefulness goes beyond the reach of this post. I will devote another entry just to talk about that and show some cool demos.

So lets focus back on the matter: As I’m using the GIO’s GDBus API for the server-side, and GDBus uses GVariant for the data packing, I needed a way to convert JavaScript data structures to GVariant and vice-versa, so that the arguments in the JavaScript context can be constructed naturally using JS native objects and arrays, and passed seamlessly to the remote APIs without the need to worry about GVariant or whatever.

Also, GVariant stores data in a binary format and there are contexts unable to handle that (like JavaScript and other scripting languages). With this API, programmers can convert their GVariants to JSON, have them processed, and then converted back to GVariant.

How does the conversion work?

Well, as you probably know, GVariant features a data-type set that is quite richer than JSON. Thus, conversion from JSON to GVariant is an ambiguous operation. To solve this, I included an optional type signature as an argument in the API, so that when a JSON tree is converted to GVariant, the signature, if provided, is used to disambiguate the data types. This mean that if a signature is provided, the resulting GVariant is guaranteed to comply with it.

GVariant * json_gvariant_deserialize (JsonNode     *json_node,
                                                         const gchar  *signature,
                                                         GError         **error);

If the signature is not provided, the conversion can still be done, and a fixed mapping is used. For detailed information on how to use the new API, you can check my build of the json-glib documentation.

I hope other people find this useful.

Fosdem? yes, me too

I’m going to Fosdem for the third year in a row. This is a great conference where hundreds of FOSS developers, users and enthusiasts gather to present their projects, hack on and discuss just about everything. The talks always cover a wide range of topics and it is normally impossible to attend all the presentations one is interested in, because many occur in parallel. The Friday Beer Event doesn’t help either, but the truth is that after some experiences in Fosdem, it is easier to survive it (or to tolerate belgium beer?). Anyway, the atmosphere at the venue, Brussels itself and the season wraps it all up to make it a very nice and worthy experience.

I thank Igalia once again for supporting me. See you there!

EventDance, a peer-to-peer inter-process communication library

EventDance is a library aiming to help developing secure and scalable peer-to-peer applications. It provides a set of easy-to-use APIs that developers can use in their programs in order to connect with other local or remote applications in a fast, secure and reliable fashion.

EventDance breaks from the idea that any system capable of running code and connect to some type of network, can be considered a “peer” and should be able to exchange data with other peers in the network under equal conditions. There is no high level concept of client or server in EventDance, but only peers that are equally important and always treated as such. For instance, a webpage running in a browser is considered a peer as much as a daemon process running in a server; and so is for software running in a mobile phone, a TV or a fridge connected to internet. This homogenized view of applications as potential creators and consumers of content and events, is a fundamental feature of EventDance and the reason behind its simplicity.

EventDance defines an abstract concept of Transport as a mean to move data between peers. Several types of transport exist and peers can choose one or more of them to communicate with another peer. Two peers can communicate as long as they have at least one transport in common. EventDance provides implementations for different types of transport (at the moment, only a Web transport). As protocols and technologies run obsolete and new ones appear, EventDance is ready to drop and embrace new types of transport; yet holding the same abstract concept. This concept is not new. Several other networking software like GNUnet have used a similar abstraction in a future-proof design.

EventDance also provides means for applications to communicate securely following the end-to-end principle. APIs are provided for programs to easily implement data privacy and peer authentication at a cryptographic level. An effort has been made to hide some of the complexity of dealing with cryto APIs as much as possible. The library uses GnuTLS as the backend for cryptography.

EventDance is written in C language using GLib/GObject. It automatically generates gobject-introspection information upon building, meaning it can be used in other languages like Javascript, Python, Lua, Vala or any other environment supporting G-I; without the need for specific bindings. The framework is designed to be lightweight, fast, scalable and energy efficient. For socket IO, it currently uses epoll exclusively, but it might support other similar mechanisms available for other platforms, in the future.

EventDance is completely event-driven, based on the GLib main loop. Applications using it are expected to be event-driven as well. All activity in the library is asynchronous and non-blocking. If some bocking operation needs to be performed, it should be moved to a thread to avoid hanging the main event loop. This is a fundamental design feature of EventDance to guarantee scalability and graceful service degradation.

EventDance is in an early stage of development. I started coding it from scratch during autumn 2009, after a first attempt written in Perl, back in 2007. I have devoted mostly my free time, and some hours sponsored by my employer, Igalia, through its hackfest initiative.

The current and only release so far is 0.1.2. Following is a list of the features currently available and others planned for the near future:

Features currently implemented:

Berkeley and Unix sockets powered by epoll (based on GIO GSocket).
Socket connection with built-in support for TLS upgrade and bandwidth/latency throttling.
Connection group for bulk management of streams (throttling, TLS, etc).
X.509 and PGP certificate objects, with support for trust validation (using GnuTLS).
Reverse proxy with load-balancing.
Web transport (with only long-polling at the moment).
Peer Manager for peer activity tracking.
Simple web server for static content.
Web selector for filtering HTTP requests.
JSON stream filter.

Features planned for next minor release (0.2):

WebSocket client and server support in the web transport.
Raw TCP transport.
UDP transport with optional PGP encryption and optional NAT traversal.
JSON-DBus bridge.
TLS agent to handle blocking cryptographic operations.
PrivateKey object.
Symmetric and asymmetric encryption/decryption.
Peer discovery mechanism.

Some of the many features planned for the long-term:

Bluetooth transport (added as an extension).
Tor transport.
Gstreamer plugin for using EventDance transports in a multimedia pipeline (added as an extension).
FastCGI web service.

EventDance is licensed under the terms of the GNU LGPL license version 3, and is hosted at Gitorious. There is neither mailing-list nor bug tracking tool at the moment, but I’m always and happily expecting feedback, criticism, suggestions and bug reports in my inbox at elima at igalia dot com. Contributions are of course very welcomed. Minimal documentation does exist, but is less than useful at the moment.

Vagalume port to Moblin2

During summer, I started to work on porting Vagalume UI to Moblin platform during my beloved hackfest hours. Vagalume is a Last.fm client for GNOME and Maemo, created by Alberto García (aka Berto). Moblin is an open source project impulsed by Intel, to create a GNU/Linux-based platform optimized for mobile devices (mainly, those powered by Intel’s Atom processor and graphics accelerator).

Vagalume’s UI is currently based on GTK+ for GNOME and Hildon for Maemo. However, Moblin user-experience is based on Nbtk and Clutter, yet it can run pure GTK+ applications. Taking a look at Nbtk toolkit was in my wishlist since it’s presentation at GCDS 2009, where some cool features of the project attracted my interest, like using CSS for styling widgets. About Clutter… well, it’s a project I personally like a lot and have been following since early versions. Motivation was ready and served.

The goal

The porting main goal is to get Vagalume running in Moblin2, sharing the platform’s look n’ feel. Revamping the UI was not just replacing widgets, but initially make Vagalume use the technologies present in a Moblin ecosystem, and “by-the-port”, perform a complete redesign of the UI, getting help from those talented designers at Moblin team.

So far…

After some weeks of hacking and festing, we have partially completed the toolkit migration, and developed a concept-proof UI redesign for the main window using Nbtk and custom Clutter widgets, except for the shell-menu which was not migrated yet . A screen-cast demo of the new UI is available. Notice that the target at this stage was not to align with Moblin’s look n’ feel, but just mixing differnt toolkits and get them to run together.

One of the new features we thought it would be interesting to include is to provide information of next and prevoius tracks, right in the main UI. The Last.fm API provides means to fetch playlist items in advance, so was not really difficult to implement that (with some core patches from Berto.. thanks).

The port’s source code is temporarily in a public repository at Gitorious.org in the moblin branch, but it will eventually get merged upstream when it gets mature enough. To build the branch, you will need a Moblin library-featured environment, like the SDK provided, or the Moblin platform itself installing the development tools and libraries. Alternatively, you can use my jhbuild moduleset.

There are some issues when building the project:

Running Nbtk in 64 bits platform drops a segfault due to a problem in libccss-0.3.1, so use 32 bits for hacking by now.
Moblin platform is not properly detected during autogen.sh stage, so you have to pass ‘–with-platform=moblin2’. This shall be fixed soon.

What’s next

Right now, we are trying to get support and feedback from Moblin UI designers, in order to plan a complete layout and style redesign. Making Vagalume UI consistent with Moblin environment is the current milestone. In the mean, we are doing several stability tests and improving toolkit integration.

Contributions are very appreciated, specially with design ideas, testing and project build integration. I can even review patches in my limited time.

Stay tuned.