Igalia – Page 2 – Bless the uncertainty

JSON and GVariant

Last week, my patch to provide GVariant integration into json-glib was merged. I filed a bug for it back in October, and after some patching iterations it finally got its way in.

This mean that now you can obtain a JSON node tree from a GVariant value and the opposite, with a single API call. I want to thank Emmanuele for reviewing my patches promptly (they were kind of lengthy), and for the positive feedback. The new API will be available in next release 0.14.

Why do we need that, anyway?

This integration seems quite natural if you give it a thought:

json-glib and GVariant are both general purpose data structure holders,
with serializing/deserializing capabilities
and both are GLib based.

My motivation to hack on that comes from my work in EventDance. I’m trying to bring D-Bus APIs into a Web page so that it is possible to connect to any server-side message bus from a page’s script and talk to the exported objects, export JavaScript objects to the bus so that they can be consumed by server applications, and own bus names. This feature and the rationale behind its usefulness goes beyond the reach of this post. I will devote another entry just to talk about that and show some cool demos.

So lets focus back on the matter: As I’m using the GIO’s GDBus API for the server-side, and GDBus uses GVariant for the data packing, I needed a way to convert JavaScript data structures to GVariant and vice-versa, so that the arguments in the JavaScript context can be constructed naturally using JS native objects and arrays, and passed seamlessly to the remote APIs without the need to worry about GVariant or whatever.

Also, GVariant stores data in a binary format and there are contexts unable to handle that (like JavaScript and other scripting languages). With this API, programmers can convert their GVariants to JSON, have them processed, and then converted back to GVariant.

How does the conversion work?

Well, as you probably know, GVariant features a data-type set that is quite richer than JSON. Thus, conversion from JSON to GVariant is an ambiguous operation. To solve this, I included an optional type signature as an argument in the API, so that when a JSON tree is converted to GVariant, the signature, if provided, is used to disambiguate the data types. This mean that if a signature is provided, the resulting GVariant is guaranteed to comply with it.

GVariant * json_gvariant_deserialize (JsonNode     *json_node,
                                                         const gchar  *signature,
                                                         GError         **error);

If the signature is not provided, the conversion can still be done, and a fixed mapping is used. For detailed information on how to use the new API, you can check my build of the json-glib documentation.

I hope other people find this useful.

Fosdem? yes, me too

I’m going to Fosdem for the third year in a row. This is a great conference where hundreds of FOSS developers, users and enthusiasts gather to present their projects, hack on and discuss just about everything. The talks always cover a wide range of topics and it is normally impossible to attend all the presentations one is interested in, because many occur in parallel. The Friday Beer Event doesn’t help either, but the truth is that after some experiences in Fosdem, it is easier to survive it (or to tolerate belgium beer?). Anyway, the atmosphere at the venue, Brussels itself and the season wraps it all up to make it a very nice and worthy experience.

I thank Igalia once again for supporting me. See you there!

EventDance, a peer-to-peer inter-process communication library

EventDance is a library aiming to help developing secure and scalable peer-to-peer applications. It provides a set of easy-to-use APIs that developers can use in their programs in order to connect with other local or remote applications in a fast, secure and reliable fashion.

EventDance breaks from the idea that any system capable of running code and connect to some type of network, can be considered a “peer” and should be able to exchange data with other peers in the network under equal conditions. There is no high level concept of client or server in EventDance, but only peers that are equally important and always treated as such. For instance, a webpage running in a browser is considered a peer as much as a daemon process running in a server; and so is for software running in a mobile phone, a TV or a fridge connected to internet. This homogenized view of applications as potential creators and consumers of content and events, is a fundamental feature of EventDance and the reason behind its simplicity.

EventDance defines an abstract concept of Transport as a mean to move data between peers. Several types of transport exist and peers can choose one or more of them to communicate with another peer. Two peers can communicate as long as they have at least one transport in common. EventDance provides implementations for different types of transport (at the moment, only a Web transport). As protocols and technologies run obsolete and new ones appear, EventDance is ready to drop and embrace new types of transport; yet holding the same abstract concept. This concept is not new. Several other networking software like GNUnet have used a similar abstraction in a future-proof design.

EventDance also provides means for applications to communicate securely following the end-to-end principle. APIs are provided for programs to easily implement data privacy and peer authentication at a cryptographic level. An effort has been made to hide some of the complexity of dealing with cryto APIs as much as possible. The library uses GnuTLS as the backend for cryptography.

EventDance is written in C language using GLib/GObject. It automatically generates gobject-introspection information upon building, meaning it can be used in other languages like Javascript, Python, Lua, Vala or any other environment supporting G-I; without the need for specific bindings. The framework is designed to be lightweight, fast, scalable and energy efficient. For socket IO, it currently uses epoll exclusively, but it might support other similar mechanisms available for other platforms, in the future.

EventDance is completely event-driven, based on the GLib main loop. Applications using it are expected to be event-driven as well. All activity in the library is asynchronous and non-blocking. If some bocking operation needs to be performed, it should be moved to a thread to avoid hanging the main event loop. This is a fundamental design feature of EventDance to guarantee scalability and graceful service degradation.

EventDance is in an early stage of development. I started coding it from scratch during autumn 2009, after a first attempt written in Perl, back in 2007. I have devoted mostly my free time, and some hours sponsored by my employer, Igalia, through its hackfest initiative.

The current and only release so far is 0.1.2. Following is a list of the features currently available and others planned for the near future:

Features currently implemented:

Berkeley and Unix sockets powered by epoll (based on GIO GSocket).
Socket connection with built-in support for TLS upgrade and bandwidth/latency throttling.
Connection group for bulk management of streams (throttling, TLS, etc).
X.509 and PGP certificate objects, with support for trust validation (using GnuTLS).
Reverse proxy with load-balancing.
Web transport (with only long-polling at the moment).
Peer Manager for peer activity tracking.
Simple web server for static content.
Web selector for filtering HTTP requests.
JSON stream filter.

Features planned for next minor release (0.2):

WebSocket client and server support in the web transport.
Raw TCP transport.
UDP transport with optional PGP encryption and optional NAT traversal.
JSON-DBus bridge.
TLS agent to handle blocking cryptographic operations.
PrivateKey object.
Symmetric and asymmetric encryption/decryption.
Peer discovery mechanism.

Some of the many features planned for the long-term:

Bluetooth transport (added as an extension).
Tor transport.
Gstreamer plugin for using EventDance transports in a multimedia pipeline (added as an extension).
FastCGI web service.

EventDance is licensed under the terms of the GNU LGPL license version 3, and is hosted at Gitorious. There is neither mailing-list nor bug tracking tool at the moment, but I’m always and happily expecting feedback, criticism, suggestions and bug reports in my inbox at elima at igalia dot com. Contributions are of course very welcomed. Minimal documentation does exist, but is less than useful at the moment.