Tag: glib

Introducing gocl, a gobject wrapper to OpenCL

For the past few months I have been working on this project to bring OpenCL closer to GNOME technologies, and today I’m glad to make the first public announcement. For the uninformed reader, OpenCL is a framework and language for writing programs that execute across heterogeneous HW pieces like CPUs, GPUs, DSPs, etc. While not applicable to any piece of software, OpenCL can unleash unparalleled performance and power efficiency on specific heavy algorithms like media decoding, cryptography, computer vision, big data indexing and processing, physics simulation, graphics, image compositing, among others.

Gocl is a GLib/GObject based library that aims at simplifying the use of OpenCL in GNOME software. It is intended to be a lightweight wrapper that adapts OpenCL programming patterns and boilerplate, and expose a simpler API that is known and comfortable to GNOME developers. Examples of such adaptations are the integration with GLib’s main loop, exposing non-blocking APIs, GError based error reporting and full gobject-introspection support. It will also be including convenient API to simplify code for the most common use patterns.

Gocl started as part of the work and research we do at Igalia on HW acceleration, that I decided to take a bit of, clean it up and release it in a way that can be useful to others. OpenCL is gaining relevance and popularity since the number of implementations and supported chips have grown significantly in recent years. Soon we are going to see OpenCL running anywhere and GNOME technologies should be ready to take advantage of it.

Full gtk-doc documentation is available, and source code is hosted at my GitHub account.

The API is very simple and limited at this stage, and should be considered very unstable. Although I’m not currently working on it full time, I do have kind of a roadmap for the API and features that I will prioritize:

  • Completing the missing asynchronous API
  • Adding API to query available OpencL extensions
  • Provide API to expose cl_khr_gl_sharing extension, for object sharing with OpenGL

You are welcome to suggest/request features that you would like to see in Gocl, as well as propose changes on the API. The GitHub issue tracking at project’s page is available for that, and also to report bugs.

So, do you know of a specific piece of software in GNOME that could potentially benefit from OpenCL? I would love to hear about it.

At Igalia, as part of our strong commitment to make the Web better and faster, we are already looking into ways of applying OpenCL to WebKit and its related technologies, and I’m personally interested on that line of work.

SHA-512 hashing support in glib

Always feels good to close old bugs, even if done unintentionally. In one of the projects I’m working on, I ran into the lack of SHA-512 support in glib and decided to step in. It turned out that such support was requested in a bug reported 3 years ago.

Whatever the reasons, the good news is that the next release of glib will ship with support for SHA-512 hashing in GChecksum. The implementation strictly follows the FIPS-180-2 standard.

Thanks to Emmanuele Bassi for reviewing my patch, Julian Andres Klode for merging it, and Igalia for sponsoring my dedication.

JSON and GVariant

Last week, my patch to provide GVariant integration into json-glib was merged. I filed a bug for it back in October, and after some patching iterations it finally got its way in.

This mean that now you can obtain a JSON node tree from a GVariant value and the opposite, with a single API call. I want to thank Emmanuele for reviewing my patches promptly (they were kind of lengthy), and for the positive feedback. The new API will be available in next release 0.14.

Why do we need that, anyway?

This integration seems quite natural if you give it a thought:

  • json-glib and GVariant are both general purpose data structure holders,
  • with serializing/deserializing capabilities
  • and both are GLib based.

My motivation to hack on that comes from my work in EventDance. I’m trying to bring D-Bus APIs into a Web page so that it is possible to connect to any server-side message bus from a page’s script and talk to the exported objects, export JavaScript objects to the bus so that they can be consumed by server applications, and own bus names. This feature and the rationale behind its usefulness goes beyond the reach of this post. I will devote another entry just to talk about that and show some cool demos.

So lets focus back on the matter: As I’m using the GIO’s GDBus API for the server-side, and GDBus uses GVariant for the data packing, I needed a way to convert JavaScript data structures to GVariant and vice-versa, so that the arguments in the JavaScript context can be constructed naturally using JS native objects and arrays, and passed seamlessly to the remote APIs without the need to worry about GVariant or whatever.

Also, GVariant stores data in a binary format and there are contexts unable to handle that (like JavaScript and other scripting languages). With this API, programmers can convert their GVariants to JSON, have them processed, and then converted back to GVariant.

How does the conversion work?

Well, as you probably know, GVariant features a data-type set that is quite richer than JSON. Thus, conversion from JSON to GVariant is an ambiguous operation. To solve this, I included an optional type signature as an argument in the API, so that when a JSON tree is converted to GVariant, the signature, if provided, is used to disambiguate the data types. This mean that if a signature is provided, the resulting GVariant is guaranteed to comply with it.

GVariant * json_gvariant_deserialize (JsonNode     *json_node,
                                                         const gchar  *signature,
                                                         GError         **error);

If the signature is not provided, the conversion can still be done, and a fixed mapping is used. For detailed information on how to use the new API, you can check my build of the json-glib documentation.

I hope other people find this useful.

EventDance, a peer-to-peer inter-process communication library

EventDance is a library aiming to help developing secure and scalable peer-to-peer applications. It provides a set of easy-to-use APIs that developers can use in their programs in order to connect with other local or remote applications in a fast, secure and reliable fashion.

EventDance breaks from the idea that any system capable of running code and connect to some type of network, can be considered a “peer” and should be able to exchange data with other peers in the network under equal conditions. There is no high level concept of client or server in EventDance, but only peers that are equally important and always treated as such. For instance, a webpage running in a browser is considered a peer as much as a daemon process running in a server; and so is for software running in a mobile phone, a TV or a fridge connected to internet. This homogenized view of applications as potential creators and consumers of content and events, is a fundamental feature of EventDance and the reason behind its simplicity.

EventDance defines an abstract concept of Transport as a mean to move data between peers. Several types of transport exist and peers can choose one or more of them to communicate with another peer. Two peers can communicate as long as they have at least one transport in common. EventDance provides implementations for different types of transport (at the moment, only a Web transport). As protocols and technologies run obsolete and new ones appear, EventDance is ready to drop and embrace new types of transport; yet holding the same abstract concept. This concept is not new. Several other networking software like GNUnet have used a similar abstraction in a future-proof design.

EventDance also provides means for applications to communicate securely following the end-to-end principle. APIs are provided for programs to easily implement data privacy and peer authentication at a cryptographic level. An effort has been made to hide some of the complexity of dealing with cryto APIs as much as possible. The library uses GnuTLS as the backend for cryptography.

EventDance is written in C language using GLib/GObject. It automatically generates gobject-introspection information upon building, meaning it can be used in other languages like Javascript, Python, Lua, Vala or any other environment supporting G-I; without the need for specific bindings. The framework is designed to be lightweight, fast, scalable and energy efficient. For socket IO, it currently uses epoll exclusively, but it might support other similar mechanisms available for other platforms, in the future.

EventDance is completely event-driven, based on the GLib main loop. Applications using it are expected to be event-driven as well. All activity in the library is asynchronous and non-blocking. If some bocking operation needs to be performed, it should be moved to a thread to avoid hanging the main event loop. This is a fundamental design feature of EventDance to guarantee scalability and graceful service degradation.

EventDance is in an early stage of development. I started coding it from scratch during autumn 2009, after a first attempt written in Perl, back in 2007. I have devoted mostly my free time, and some hours sponsored by my employer, Igalia, through its hackfest initiative.

The current and only release so far is 0.1.2. Following is a list of the features currently available and others planned for the near future:

Features currently implemented:

  • Berkeley and Unix sockets powered by epoll (based on GIO GSocket).
  • Socket connection with built-in support for TLS upgrade and bandwidth/latency throttling.
  • Connection group for bulk management of streams (throttling, TLS, etc).
  • X.509 and PGP certificate objects, with support for trust validation (using GnuTLS).
  • Reverse proxy with load-balancing.
  • Web transport (with only long-polling at the moment).
  • Peer Manager for peer activity tracking.
  • Simple web server for static content.
  • Web selector for filtering HTTP requests.
  • JSON stream filter.

Features planned for next minor release (0.2):

  • WebSocket client and server support in the web transport.
  • Raw TCP transport.
  • UDP transport with optional PGP encryption and optional NAT traversal.
  • JSON-DBus bridge.
  • TLS agent to handle blocking cryptographic operations.
  • PrivateKey object.
  • Symmetric and asymmetric encryption/decryption.
  • Peer discovery mechanism.

Some of the many features planned for the long-term:

  • Bluetooth transport (added as an extension).
  • Tor transport.
  • Gstreamer plugin for using EventDance transports in a multimedia pipeline (added as an extension).
  • FastCGI web service.

EventDance is licensed under the terms of the GNU LGPL license version 3, and is hosted at Gitorious. There is neither mailing-list nor bug tracking tool at the moment, but I’m always and happily expecting feedback, criticism, suggestions and bug reports in my inbox at elima at igalia dot com. Contributions are of course very welcomed. Minimal documentation does exist, but is less than useful at the moment.