Carlos Garcia Campos

As every year many ideas came up during the WebKitGTK+ hackfest presentation, but this time there was one we all were very excited about, the multiple web processes support. Apple developers already implemented the support for multiple web processes in WebKit, which is mostly cross platform, but it requires the network process support to properly work (we need a common network process where cookies, HTTP cache, etc are shared for all web processes in the same web context). Soup based WebKit ports don’t implement the network process yet, so the goal of the hackfest became to complete the network process implementation previously started by EFL and Nix guys, as a first step to enable the multiple web processes support. Around 10 people were working on this goal during the whole hackfest, meeting from time to time to track the status of the tasks and assigning new ones.

• Andy updated some of the patches already waiting for a review and set up github repo with all those patches so that we all could work without dealing with individual patches.
• Sergio updated the patch proposed by Balazs to implement the bits required to connect the network process with the web process using UNIX domain sockets.
  
• Brian added the option to build with the network process enabled (disabled by default).
• Martin updated the soup backend to use WebKit buffers when reading network data instead of char pointers because the network process only accepts WebKit buffers.
• Kwang and Brian fixed the SSL errors policy handling when using the network process.
• Berto fixed the way the network process was enabled, by moving it to the right place and enabling it by default when WebKit is build with network process support.
• Kwang fixed a method in the web context to send messages to the current networking process to allow sending messages to the network process even when using the single shared process model.
• Berto added support to run the network process with an arbitrary prefix command in debug builds for easily debugging crashes or leaks.
• Gustavo fixed the original encoded data size calculation, since we were using the decoded data size instead.
• Kwang updated the patch he started and Csaba continued to add support for cache model in the network process.

After all this awesome work we managed to have the basic support, with MiniBrowser perfectly rendering pages and allowing navigation using the network process. But as expected, there were some bugs and missing features, so I ran the WebKit2 unit tests and we took failing tests to investigate why they were failing and how to fix them.

• Berto is working on loader client tests failing due to missing notifications in the network process.
• Adrian submitted a patch to add new API to select the process model that will be used when the network process support is finished. He is also working on the problems of having multiple instances of the web extensions.
• Kwang is working on fixing the cookies handling in the network process.
• I submitted a patch to implement the initiating page support in the remote networking context used by the network process. I’m working on adding support for custom URI schemes to the network process as well.

So, we are actually far from having a complete and stable network process support, but it’s a huge step forward. The good news is that once we have network process implemented, the multiple web processes support will work automatically just by selecting the multiple web process model.

All this sounds like a lot of work done, but that’s only a small part of what has happened this week in Coruña:

• Martin and Gustavo made more parts of WebKit actually build with the cmake build.
• Jon made several improvements in Epiphany UI.
• Gustavo fixed the default charset encoding used by Epiphany.
• Iago, Edu made some progress in the wayland support for WebKit2.
• Dan was working on HTTP2 implementation for libsoup.
• Zan was finalizing his GSoC work under Martin’s mentorship to bring WebGL support under Wayland
• Gustavo added support for right-side docking of the web inspector in WebKitGTK+.
• Javi, Rego and Minhea were focused on a new implementation for the selections in CSS regions.
• Calvaris began to rewrite the GTK media controls once more, this time in JavaScript.
• Zan finished the patches to add battery support in WebKitGTK+ using upower.
• Martin, Gustavo and Zan worked on support for testing WebGL and accelerated compositing layout tests in WebKitGTK+.
• Brian, Alex and Zan were working on the parsing of the valgrind xml output used when running the tests under valgrind to detect memory leaks.
• Brendan added a setting to both WebKit1 and WebKit2 APIs to enable media source and fixed a crash in several video layout tests.
• Claudio went back to his work on the notifications support for WebKitGTK+ and was reviewing patches like crazy.
• Philippe worked on the WebRTC implementation for the GStreamer WebKit media backend.
• Mario, Joanie and API were focused on accessibility, also making sure that the multiple web processes doesn’t affect the accessibility support.
• Brendan also worked on MediaSource, investigating how to handle video resolution changes.
• I removed all the WebKit1 unused code from Epiphany, and moved the GNOME shell search provider to its own binary.

And I’m sure I’m missing more great stuff done that I could not follow closely. It’s definitely been a very productive hackfest that it would haven’t been possible without the sponsors, Igalia and the GNOME Foundation. Thanks!

The multiprocess architecture of WebKit2 brought us a lot of advantages, but it also introduced important challenges, like how to expose some features that now live in the Web Process (DOM, JavaScript, etc.). The UI process API is fully asynchronous to make sure the UI is never blocked, but some APIs like the DOM bindings are synchronous by design. To expose those features that live in the Web Process, WebKit2GTK+ provides a Web Extensions mechanism. A Web Extension is like a plugin for the Web Process, that is loaded at start up, similar to a GTK module or gio extension, but that runs in the Web Process. WebKit2GTK+ exposes a simple low level API that at the moment provides access to three main features:

• GObject DOM bindings: The exactly same API used in WebKit1 is available in WebKit2.
• WebKitWebPage::send-request signal: It allows to change any request before it is sent to the server, or even simply prevent it from being sent.
• Custom JavaScript injection: It provides a signal, equivalent to WebKitWebView::window-object-cleared in WebKit1, to inject custom JavaScript using the JavaScriptCore API. (Since 2.2)

This simple API doesn’t provide any way of communication with the UI Process, so that the user can use any IPC mechanism without interfering with the internal WebKit IPC traffic. Epiphany currently installs a Web Extension to implement some of its features such us pre-filled forms, ads blocker or Do Not Track using D-BUS for the communication between the Web Extension and the UI Process.

How to write a Web Extension?

Web Extensions are shared libraries loaded at run time by the Web Process, so they don’t have a main function, but they have an entry point called by the WebProcess right after the extension is loaded. The initialization function must be called webkit_web_extension_initialize() and it receives a WebKitWebExtension object as parameter. It should also be public, so make sure to use the G_MODULE_EXPORT macro. This is the function to initialize the Web Extension and can be used, for example, to be notified when a web page is created.

static void
web_page_created_callback (WebKitWebExtension *extension,
                           WebKitWebPage      *web_page,
                           gpointer            user_data)
{
    g_print ("Page %d created for %s\n", 
             webkit_web_page_get_id (web_page),
             webkit_web_page_get_uri (web_page));
}

G_MODULE_EXPORT void
webkit_web_extension_initialize (WebKitWebExtension *extension)
{
    g_signal_connect (extension, "page-created", 
                      G_CALLBACK (web_page_created_callback), 
                      NULL);
}

This would be a minimal Web Extension, it does nothing yet, but it can be compiled and loaded so let’s see how to create a Makefile.am file to build the extension.

webextension_LTLIBRARIES = libmyappwebextension.la
webextensiondir = $(libdir)/MyApp/web-extension
libmyappwebextension_la_SOURCES = my-app-web-extension.c
libmyappwebextension_la_CFLAGS = $(WEB_EXTENSION_CFLAGS)
libmyappwebextension_la_LIBADD = $(WEB_EXTENSION_LIBS)
libmyappwebextension_la_LDFLAGS = -module -avoid-version -no-undefined

The extension will be installed in $(libdir)/MyApp/web-extension so we need to tell WebKit where to find web extensions before the Web Process is spawned. Call webkit_web_context_set_web_extensions_directory() as soon as possible in your application, before any other WebKit call to make sure it’s called before a Web Process is launched. You can create a preprocessor macro in the Makefile.am to pass the value of the Web Extensions directory.

myapp_CPPFLAGS = -DMYAPP_WEB_EXTENSIONS_DIR=\""$(libdir)/MyApp/web-extension"\"

And then in the code

webkit_web_context_set_web_extensions_directory (webkit_web_context_get_default (), 
                                                 MYAPP_WEB_EXTENSIONS_DIR);

The Web Extension only needs WebKit2GTK+ to build, so in the configure.ac you can define WEB_EXTENSION_CFLAGS and WEB_EXTENSION_LIBS using pkg-config macros.

PKG_CHECK_MODULES(WEB_EXTENSION, [webkit2gtk-3.0 >= 2.0.0])
AC_SUBST(WEB_EXTENSION_CFLAGS)
AC_SUBST(WEB_EXTENSION_LIBS)

This should be enough. You should be able to build and install the Web Extension with you program and see the printf message every time a page is created. But that’s a useless example, let’s see how to use the Web Extensions API to do something useful.

Accessing the DOM

The GObject DOM bindings API available in WebKit1 is also exposed in WebKit2 from the Web Extensions API. We only need to call webkit_web_page_get_dom_document() to get the WebKitDOMDocument of the given web page.

static void
web_page_created_callback (WebKitWebExtension *extension,
                           WebKitWebPage      *web_page,
                           gpointer            user_data)
{
    WebKitDOMDocument *document;
    gchar             *title;

    document = webkit_web_page_get_dom_document (web_page);
    title = webkit_dom_document_get_title (document);
    g_print ("Page %d created for %s with title %s\n", 
             webkit_web_page_get_id (web_page),
             webkit_web_page_get_uri (web_page),
             title);
    g_free (title);
}

Using WebKitWebPage::send-request signal

Using the Web Extensions API it’s possible to modify the request of any resource before it’s sent to the server, adding HTTP headers or modifying the URI. You can also make WebKit ignore a request, for example to block resources depending on the URI, by simply connecting to the signal and returning TRUE.

static gboolean
web_page_send_request (WebKitWebPage     *web_page,
                       WebKitURIRequest  *request,
                       WebKitURIResponse *redirected_response,
                       gpointer           user_data)
{
    const char *request_uri;
    const char *page_uri;

    request_uri = webkit_uri_request_get_uri (request);
    page_uri = webkit_web_page_get_uri (web_page);

    return uri_is_an_advertisement (request_uri, page_uri);
}

static void
web_page_created_callback (WebKitWebExtension *extension,
                           WebKitWebPage      *web_page,
                           gpointer            user_data)
{
    g_signal_connect_object (web_page, "send-request",
                             G_CALLBACK (web_page_send_request),
                             NULL, 0);
}

Extending JavaScript

Using the JavaScriptCore API it’s possible to inject custom JavaScript code by connecting to the window-object-cleared signal of the default WebKitScriptWorld. You can get the global JavaScript execution context by calling webkit_frame_get_javascript_context_for_script_world() for the WebKitFrame passed as parameter of the window-object-cleared signal.

static void 
window_object_cleared_callback (WebKitScriptWorld *world, 
                                WebKitWebPage     *web_page, 
                                WebKitFrame       *frame, 
                                gpointer           user_data)
{
    JSGlobalContextRef jsContext;
    JSObjectRef        globalObject;

    jsContext = webkit_frame_get_javascript_context_for_script_world (frame, world);
    globalObject = JSContextGetGlobalObject (jsContext);

    /* Use JSC API to add the JavaScript code you want */
}

G_MODULE_EXPORT void
webkit_web_extension_initialize (WebKitWebExtension *extension)
{
    g_signal_connect (webkit_script_world_get_default (), 
                      "window-object-cleared", 
                      G_CALLBACK (window_object_cleared_callback), 
                      NULL);
}