FileTea, low friction anonymous file sharing

Posted by elima on September 01, 2011


Let me present you FileTea, a project enabling anonymous file sharing for the Web. It is designed to be simple and easy to use, to run in (modern) browsers without additional plugins, and to avoid the hassle of user registration.

Works like this:

  • Mary wants to share a file with Jane. She opens a browser and navigates to a FileTea service.
  • Mary drag-and-drops the file (or files) she wants to send into the page and copies the short url generated for each file.
  • Mary sends the url to Jane by instant messaging, e-mail, SMS or just posts it somewhere.
  • Jane receives the url, opens it in her browser and downloads the file.

A reference deployment is running at http://filetea.me. It is alpha version and is kindly hosted by Igalia (thanks!). Please, feel free to try it (and provide feedback!).

FileTea is free software and released under the terms of the GNU AGPL license. That means that you can install it in your own server and host it yourself for your organization, your business or your friends. The source code is hosted at Gitorious.

We are not alone

There are similar services running out there like http://min.us, http://ge.tt, http://imm.io, and maybe others I’m not aware of. But FileTea is different in three important aspects:

  • FileTea is free software (including server side), meaning development is open to the community, and you can run your own instance and make it better.
  • FileTea does not store any file in the server. It just synchronizes and bridges an upload from the seeder with a download to the leacher.
  • FileTea sets no limit to the size of shared files.

Features, my friend

Currently, FileTea has the bare minimum features to allow file-sharing, but I have a long list of possibly cool stuff to add as I find free time to work on them. Some hot topics are:

  • Global and per-transfer bandwidth limitation (up and down).
  • Proper thumbnails for the shared files.
  • Bulk sharing: the ability to share a group of files under a single url and download a zip or tar ball with all files together.

If you have more ideas, I would be happy to hear them.

So that’s it, happy sharing!

EventDance 0.1.4 released

Posted by elima on May 02, 2011

During Easter holidays, I finally managed to find time to close EventDance 0.1.3 development cycle and release 0.1.4. This milestone took more than expected for several reasons, mainly due to some last minute API changes I had to introduce and a couple of features I couldn’t resist to implement earlier. The result is a long changelog that I will try to summarize:

  • Basic API for asymmetric (public-key) cryptography

    EvdPkiPubkey and EvdPkiPrivkey classes provide abstraction for PKI public and private key respectively. They basically are asynchronous, GIO-friendly wrappers for libgcrypt PK functions. There is also API for asynchronous key-pair generation. By now, only encryption/decryption using RSA algorithm is supported.

  • Basic API for symmetric cryptography

    EvdTlsCipher also provides an asynchronous, GIO-friendly wrapper for libcrypt symmetric crypto API, adding some nice features like data auto-padding and key aligning built right in. Not all algorithms supported by libgcrypt are available but only the most popular (e.g, AES 128/192/256, ARCFOUR).

  • SNI and lazy certificate selection for TLS credentials

    Server Name Indication is a SSL/TLS extension that permits a client to request the domain name before the certificate is committed to the server. This feature is available in GnuTLS and is now exported to EvdTlsSession. Also, EvdTlsCredentials added a callback to select the certificate to send to the peer during the TLS handshake. The combination of these two features is critical to implement an SSL/TLS capable reverse Web proxy. I’m seriously considering to include one such proxy inside EventDance, that would export a D-Bus API over the system bus to allow applications to easily add/remove virtual hosts and server backends on-the-fly.

  • Websockets mechanism into EvdWebTransport

    Now the web transport negotiates mechanism with the browser during handshake and uses websockets if supported, otherwise falls back to long-polling. Only version 76 (hybi-00) of the spec is implemented so far.

  • EvdDBusBridge

    A component to connect a web page to a D-Bus message bus running in the server, allowing client-side Web applications to proxy/export objects and acquire bus names. Check my previous post introducing this feature for details

  • EvdJsonrpc

    An asynchronous, GIO-friendly implementation of the JSON-RPC protocol version 1.0, specifically designed to work well with EventDance transports.

  • EvdDaemon

    An abstraction for any program that runs as a service daemon. The purpose is that if you are implementing a daemon, you just use an EvdDaemon instance and automagically get an event-loop (GMainLoop), pid-file management, syslog-based logging, daemonizing (console detaching) and clean program termination. The pid-file and syslog functionalities are still on the way though.

  • EvdDBusDaemon

    A component that launches a custom D-Bus message bus and tracks its execution. This is useful when an application needs to use a custom message bus instead of the well-known ones; for security or sandboxing reasons.

Also, as usual, lots of bugfixes and random improvements. A dependency on json-glib was added too.

Now and for the next weeks, I’m running a documentation and annotations sprint, something I have delayed too much already. I will also write a couple of basic tutorials on how to build and use EventDance. Stay tuned.

The Web jumps into D-Bus

Posted by elima on March 19, 2011

Lets start with the fun. The following demos show the EventDance’s D-Bus bridge in action:

Proxying org.freedesktop.Notifications

Directly from a Web page, we create a proxy to the standard freedesktop.org object /org/freedesktop/Notifications which is exported to the session bus running in my normal desktop environment. Then, we call the remote method Notify using a simple web form. This method opens a notification dialog showing a quick note to the desktop user. The proxy also watches the remote signal NotificationClosed telling that a notification dialog was closed.
During the demo we open different browsers to show the cross-browser nature of EventDance’s Web transport.

Your browser doesn’t seem to support HTML5 video tag but you can download the video directly.

Screencast 1: D-Bus proxy example
Acquiring a bus name and exporting an object

In the next screencast we acquire/release a name on the session bus, and export a simple object onto it. The object has one method Ask that receives a string as input argument, shows a confirmation dialog on the Web page, and return a boolean value depending on whether the user pressed “Ok” or “Cancel” to close the dialog. We use D-Feet to call the remote method on the browser that owns the name at the moment of the call.

Your browser doesn’t seem to support HTML5 video tag but you can download the video directly.

Screencast 2: D-Bus name owning example
The client-side source code of both demos is available at examples/common/dbus-bridge-proxy.html and examples/common/dbus-bridge-own-name.html respectively, inside EventDance repository. You can check them to realize how simple is the API to connect to and use the D-Bus connection.

The server-side code for both demos is the same, and is public at examples/dbus-bridge.c. There is also a JavaScript version at examples/js/dbusBridge.js and a python version at examples/python/dbus-bridge.py. To run JavaScript and python examples you need EventDance compiled with –enable-introspection configure flag active (the default), and GJS or pygobject respectively available in your system.

The D-Bus bridge

At the highest level, EventDance library provides a set of transports and IPC mechanisms. The D-Bus bridge is just one of these mechanisms, written specifically to work over the Web transport. Its purpose is to connect a Web page with a remote D-Bus daemon running on the Web server, using the EventDance’s Web transport (long-polling or websocket if available).

D-Bus uses a binary protocol, and the browser’s JavaScript context doesn’t (yet) support binary data. Thus, it was sensible to translate D-Bus messages to a browser-friendly protocol, in this case JSON. Using the API recently added into json-glib to integrate GVariant and JSON, the bridge translates messages back and forth from GDBus to JSON and vice-versa. Also, it tries to reduce message payload as much as possible, by caching the GDBus objects server-side, and sending only their references over the wire. This makes the protocol very simple and lightweight.

Cool, now what?

Hopefully you are already wondering about the many purposes this feature can serve.

While many people envision that the next desktop will be the browser, many more do use Web applications almost exclusively, already today. The traditional separation between Web and Desktop app development is blurring. Browsers have become powerful platforms for running complex applications, and this situation is speeding up with the broad and increasing adoption of HTML5 standards by major browsers.

On the other hand, we have D-Bus, a freedesktop.org standard that is at the core of almost every GNU/Linux system out there. It is the de-facto IPC mechanism on which your applications talk and share. D-Bus allows us to write a program in any language, and export its usefulness over a standard channel. Also allows us to write differentiated UIs (e.g, Qt vs. GTK+ vs. NCurses) to interface a common functionality. Yes, one bus to bind them all!

Joining these two pieces together is just the next logical step. A step towards bringing together the best of two contexts: the ubiquity of the Web and the inter-process collaborative nature of the Desktop.

We need to write applications that you can host and use securely and reliably not only in your computer, but anywhere in the Planet where you happen to have a browser plugged into the Net; whether it is your laptop, mobile phone, tablet or your neighbors’ PS3. We also need to encourage application developers to export the logic of their programs over D-Bus, to allow other platforms (like the Web!) to reuse it. Telepathy is a good example of such program.

Almost identical applications in terms of functionality are written for many FOSS environments like GNOME, KDE, MeeGo, Android, etc; yet many times only the user experience and the technologies used to build it are different. Although not always possible, there is room for a wider code-reusing culture if we come back to the original Unix philosophy:


Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

That universal interface has just been upgraded.

JSON and GVariant

Posted by elima on January 31, 2011

Last week, my patch to provide GVariant integration into json-glib was merged. I filed a bug for it back in October, and after some patching iterations it finally got its way in.

This mean that now you can obtain a JSON node tree from a GVariant value and the opposite, with a single API call. I want to thank Emmanuele for reviewing my patches promptly (they were kind of lengthy), and for the positive feedback. The new API will be available in next release 0.14.

Why do we need that, anyway?

This integration seems quite natural if you give it a thought:

  • json-glib and GVariant are both general purpose data structure holders,
  • with serializing/deserializing capabilities
  • and both are GLib based.

My motivation to hack on that comes from my work in EventDance. I’m trying to bring D-Bus APIs into a Web page so that it is possible to connect to any server-side message bus from a page’s script and talk to the exported objects, export JavaScript objects to the bus so that they can be consumed by server applications, and own bus names. This feature and the rationale behind its usefulness goes beyond the reach of this post. I will devote another entry just to talk about that and show some cool demos.

So lets focus back on the matter: As I’m using the GIO’s GDBus API for the server-side, and GDBus uses GVariant for the data packing, I needed a way to convert JavaScript data structures to GVariant and vice-versa, so that the arguments in the JavaScript context can be constructed naturally using JS native objects and arrays, and passed seamlessly to the remote APIs without the need to worry about GVariant or whatever.

Also, GVariant stores data in a binary format and there are contexts unable to handle that (like JavaScript and other scripting languages). With this API, programmers can convert their GVariants to JSON, have them processed, and then converted back to GVariant.

How does the conversion work?

Well, as you probably know, GVariant features a data-type set that is quite richer than JSON. Thus, conversion from JSON to GVariant is an ambiguous operation. To solve this, I included an optional type signature as an argument in the API, so that when a JSON tree is converted to GVariant, the signature, if provided, is used to disambiguate the data types. This mean that if a signature is provided, the resulting GVariant is guaranteed to comply with it.

GVariant * json_gvariant_deserialize (JsonNode     *json_node,
                                                         const gchar  *signature,
                                                         GError         **error);

If the signature is not provided, the conversion can still be done, and a fixed mapping is used. For detailed information on how to use the new API, you can check my build of the json-glib documentation.

I hope other people find this useful.

EventDance, a peer-to-peer inter-process communication library

Posted by elima on October 14, 2010

EventDance is a library aiming to help developing secure and scalable peer-to-peer applications. It provides a set of easy-to-use APIs that developers can use in their programs in order to connect with other local or remote applications in a fast, secure and reliable fashion.

EventDance breaks from the idea that any system capable of running code and connect to some type of network, can be considered a “peer” and should be able to exchange data with other peers in the network under equal conditions. There is no high level concept of client or server in EventDance, but only peers that are equally important and always treated as such. For instance, a webpage running in a browser is considered a peer as much as a daemon process running in a server; and so is for software running in a mobile phone, a TV or a fridge connected to internet. This homogenized view of applications as potential creators and consumers of content and events, is a fundamental feature of EventDance and the reason behind its simplicity.

EventDance defines an abstract concept of Transport as a mean to move data between peers. Several types of transport exist and peers can choose one or more of them to communicate with another peer. Two peers can communicate as long as they have at least one transport in common. EventDance provides implementations for different types of transport (at the moment, only a Web transport). As protocols and technologies run obsolete and new ones appear, EventDance is ready to drop and embrace new types of transport; yet holding the same abstract concept. This concept is not new. Several other networking software like GNUnet have used a similar abstraction in a future-proof design.

EventDance also provides means for applications to communicate securely following the end-to-end principle. APIs are provided for programs to easily implement data privacy and peer authentication at a cryptographic level. An effort has been made to hide some of the complexity of dealing with cryto APIs as much as possible. The library uses GnuTLS as the backend for cryptography.

EventDance is written in C language using GLib/GObject. It automatically generates gobject-introspection information upon building, meaning it can be used in other languages like Javascript, Python, Lua, Vala or any other environment supporting G-I; without the need for specific bindings. The framework is designed to be lightweight, fast, scalable and energy efficient. For socket IO, it currently uses epoll exclusively, but it might support other similar mechanisms available for other platforms, in the future.

EventDance is completely event-driven, based on the GLib main loop. Applications using it are expected to be event-driven as well. All activity in the library is asynchronous and non-blocking. If some bocking operation needs to be performed, it should be moved to a thread to avoid hanging the main event loop. This is a fundamental design feature of EventDance to guarantee scalability and graceful service degradation.

EventDance is in an early stage of development. I started coding it from scratch during autumn 2009, after a first attempt written in Perl, back in 2007. I have devoted mostly my free time, and some hours sponsored by my employer, Igalia, through its hackfest initiative.

The current and only release so far is 0.1.2. Following is a list of the features currently available and others planned for the near future:

Features currently implemented:

  • Berkeley and Unix sockets powered by epoll (based on GIO GSocket).
  • Socket connection with built-in support for TLS upgrade and bandwidth/latency throttling.
  • Connection group for bulk management of streams (throttling, TLS, etc).
  • X.509 and PGP certificate objects, with support for trust validation (using GnuTLS).
  • Reverse proxy with load-balancing.
  • Web transport (with only long-polling at the moment).
  • Peer Manager for peer activity tracking.
  • Simple web server for static content.
  • Web selector for filtering HTTP requests.
  • JSON stream filter.

Features planned for next minor release (0.2):

  • WebSocket client and server support in the web transport.
  • Raw TCP transport.
  • UDP transport with optional PGP encryption and optional NAT traversal.
  • JSON-DBus bridge.
  • TLS agent to handle blocking cryptographic operations.
  • PrivateKey object.
  • Symmetric and asymmetric encryption/decryption.
  • Peer discovery mechanism.

Some of the many features planned for the long-term:

  • Bluetooth transport (added as an extension).
  • Tor transport.
  • Gstreamer plugin for using EventDance transports in a multimedia pipeline (added as an extension).
  • FastCGI web service.

EventDance is licensed under the terms of the GNU LGPL license version 3, and is hosted at Gitorious. There is neither mailing-list nor bug tracking tool at the moment, but I’m always and happily expecting feedback, criticism, suggestions and bug reports in my inbox at elima at igalia dot com. Contributions are of course very welcomed. Minimal documentation does exist, but is less than useful at the moment.