Category Archives: weston

Chromium Mus/Ozone update (H1/2017): wayland, x11

Since January, Igalia has been working on a project whose goal is to make the latest Chromium Browser able to run natively on Wayland-based environments. The project has various phases, requires us to carve out existing implementations and align our work with the direction Chromium’s mainline is taking.

In this post I will provide an update on the progresses we have made over 2017/H1, as well as our plans coming next.

In order to jump straight to the latest results section (including videos) without the details, click here.

Background

In 2016/Q4, my fellow Igalian Frédéric Wang and I ran a warm-up project to check the status of the existing Wayland support in Chromium’s mainline repository, and estimate how much work was needed to get the full (and latest) Chromium browser running on Wayland.

As part of this warm-up we were able to build and launch ChromeOS’s Chrome for both desktop and embedded Linux distributions, featuring either X11 or Wayland. Automotive Grade Linux running on the Renesas’ R-car M3 board is an example of the embedded environments we tested.

Mus+ash on LinuxOS (Nov/2016).
Mus+ash on LinuxOS (Nov/2016)

Although this was obviously not our end goal (some undesirable ChromeOS widgets were visible at the bottom), it allowed us to verify the overall performance of the build, and experiment with things a bit. Here is a brief summary of the most relevant findings:

  • There is an ongoing effort to reimplement the Ash (aka “cash” or classic Ash) Window Manager on top of the new UI Service (aka Mus). It is the so called mus+ash project, a ChromeOS oriented effort that is the continuation of Mandoline project.

  • It is possible to build mus+ash for various platforms including Linux, ChromeOS and Windows. On Linux specifically, it is possible to make off-device ChromeOS builds of mus+ash, and run it on desktop Linux for testing purposes. A more minimalistic Window Manager version is also available in //mash/simple_wm, and should run on regular Linux builds too.

  • mus+ash can be built with Ozone enabled. This means that it can run with the various backends Ozone has. It is worth saying that the upstream focus seems to be the Ozone/DRM-GBM backend, for ChromeOS.

  • Ozone itself has morphed over time from an abstraction layer underneath the Aura toolkit, to be a layer underneath Mus.

Last, we could publish some worth reading content:

2017 developments

At the beginning of this new phase of the project, we knew we needed to work on two different levels, in order to have the Chromium browser running on desktop Linux, ideally without functionality losses if compared against the stock Chromium browser on X11: both Mus and Ozone needed to support ‘external window’ mode.

For the sake of completeness, the term external window mode above is the terminology we chose to represent a regular desktop application on Linux, where the host Window Manager takes care of windowing actions like maximize, minimize, restore and fullscreen the application window. Also, the application itself reacts to content size changes accordingly. Analogously, when we say an application runs in internal window mode, it runs within the (M)ash shell environment, the builtin Window Manager that powers ChromeOS builds. Applications in this mode do not interact with the host WM.

A huge pro about how mus+ash is being implemented is that the Chrome browser itself already works as it ought to in non-ChromeOS Mus-based environments: either we are running Mus in internal or external window modes, Chrome will work just like Chrome for a Linux desktop ought to.

That being said, we identified the following set of tasks, on both Ozone and Mus sides.

Ozone tasks:

Extend Ozone so that both Window Manager provided window decorations (like a regular X11 window on Ubuntu) and Chromium’s builtin window decoration work flawlessly.
On Wayland, window decorations can be provided either by the client side (application), or by the Wayland server (compositor). The fact that Weston does not provide window decorations by default, forces us to support Chromium’s builtin one for the good.
In case of the Chromium’s builtin window decorations …
… add support for basic windowing functionality like maximize, minimize, restore and fullscreen, as well as window dragging and resizing.
Add support to “window close”.
In internal window mode, there is no concept of window closing, because the outer/native Ozone window represents the Window Manager display, which is not supposed to get closed. In external window mode, windows can be closed freely, as per the needs of the user.
Add support for multi window browsing.
Each browser window should be backed by its own acceleratedWidget. This also includes being able to draw widgets that on stock Linux/X11 builds use native windows: tooltips, (nested and context) menus.
Handle keyboard focus activation when switching windows.
Again in ‘internal window’ mode the outer/native Ozone window is unique and represents the Window Manager display, not losing or gaining focus at any time. Focus switching of inner windows is handled by mus+ash. In ‘external window’ mode, user can open as many Browser windows as he wants, and focus switches at the Window Manager level, should reflect on the application focus.

Mus tasks:

Fix assumptions that make sense for mus+ash on ChromeOS only.
The fact that a display::Display instance mapped always to a single ui::ws::Display instance.
Ownership model
Some Mus objects have slightly different ownership in external window mode: ws::Display, ws::WindowManagerState, ws::WindowManagerDisplayRoot and ws::WindowTree
The plan

After meeting with rjkroege@ at BlinkOn 7, we defined a highlevel plan to tackle the project. These were the main action points:

1) Extend the mus_demo to work in ‘external window’ mode.
2) Start fixing 1:1 assumptions in the code, e.g. display::Display ui::ws::Display.
3) Extend Mus to work on ‘external window’ mode.
4) Extend Ozone to work on ‘external window’ mode.
5) Make the code that handles the existing –mus command line parameter non-ChromeOS specific.

With this 5 highlevel steps done, we would be able to get Chrome/Mus running on desktop Linux, on the various Ozone backends.

The action
Mus Demo

We were able to get mus_demo working in ‘external window’ mode, by making use of the existing WindowTreeHostFactory API.

1:1 assumptions

Although WindowTreeHostFactory was in place for the creation WindowTreeHost instances, both Mus and Ozone still had assumptions that only applied in a ChromeOS context. The Googler kylechar@ jumped in and fixed some of them, helping out on our effort.

Mus and Ozone carve out

In order to get the 3rd and 4th steps going, we decided to switch our main development repository to a GitHub fork, so that we could expedite reviews and progresses. Given Igalia’s excellence in carrying downstream forks of large projects forward, we established a contribution process and a rebase strategy that would allow us to move at a good pace, and still stay as close as possible to Chromium’s tip of trunk.

These are some of the main changes in our downstream repository:

  • Mus and Aura/Mus were changed to support an ‘external window’ mode flow: there is a single WindowTreeClient (Aura/Mus) instance that maps to another single WindowTree instance (Mus). The relationship is invariable and is ensured by ui::ws::WindowTreeHostFactoryRegistrar class.

  • In this new set up, ui::ws::WindowTreeHostFactory::CreatePlatformWindow can create as many WindowTreeHost / ui::ws::Display instances as needed. ui::ws::Display triggers creation of PlatformDisplay objects, which hold Ozone window handles. Hence, every Chromium window (and some browser widgets) gets backed by its own acceleratedWidget.

  • In mus+ash, there are some operations accomplished through a cooperation between both Mus and Ash, or Mus and Aura/Mus sides. For example, setting “frame decorations” values in mus+ash goes through the following path:

    1) ash::mus::WindowManager get frame decoration values as per the “material design” in use and passes it to aura::WindowTreeClient::SetFrameDecorationValues.
    2) WindowTree::WmSetFrameDecorationValues
    3) WindowManagerState::SetFrameDecorationValues
    4) UserDisplayManager::OnFrameDecorationValuesChanged
    5) ScreenMus::OnDisplays()
    6) These values are used later on to draw “non client frame” area of the Browser window, which “frame” that contains the Web contents area.

    On Chrome/Mus LinuxOS, we skip this round trip by using the same “non client frame view” as stock Linux/X11 Chrome: OpaqueBrowserFrameView.

  • In mus+ash all Browser widgets creation take the DesktopNativeWidgetAura path. This implies a new WindowPort and new WindowTreeHost instances per widget. Adding support for this in Mus and Ozone sides would require lots of work and refactory. Hence, we again decided to use the stock Linux/X11 flow: for widgets currently backed by a native window (tooltips, menus) we use the NativeWidgetAura path, whereas for others widgets (bookmark banner and zoom in/out banners, URL completion window, status bubble, etc) we use NativeWidgetAura. Also, this choice made extending Ozone accordingly simpler.

Status and next steps

We have reached a point where we can show Chrome Ozone/Mus on desktop Linux, on using both X11 and Wayland backends, and here is how it is looking like today:

Wayland:

X11:

The –mus and –ozone-platform={name} command line parameters control the Chrome configuration. Please note that the same Chrome binary is used.

Some of our next steps for Chromium Mus/Ozone are:

  • Continue to fix the windowing features (namely window resize and dragging, as well as drag and drop) when Chromium’s builtin window decorations are used.
  • Provide updated yocto builds on Igalia’s meta-browser fork.
  • Support newer shell protocols like XDG v6, supported by Fedora 25.
  • Ensure no feature losses when compared to stock Chromium X11/Linux.
  • Ensure there is no performance penalties when compared to stock Chromium X11/Linux.
  • Start to upstream some of the changes.

We are also considering providing prebuilt binaries, so that earlier adopters can test the status.

This project is sponsored by Renesas Electronics …

renesas_logomark_l

… and is being performed by Igalian hacker Maksim Sisov and Antonio Gomes (me) on behalf of Igalia, being Frederic Wang an emeritus contributor.

igalia-logo-364x130

Chromium, ozone, wayland and beyond

Over the past 2 months my fellow Igalian Frédéric Wang and I have been working on improving the Wayland support in Chromium. I will be detailing on this post some of the progresses we’ve made and what our next steps are.

Background

The more Wayland evolved as a mature alternative to the traditional X11-based environments, the more the Chromium community realized efforts were needed to run Chrome natively and flawlessly on such environments. On that context, the Ozone project began back in 2013 with the main goal of making porting Chrome to different graphics systems easier.

Ozone consists in a set of C++ classes for abstracting different window systems on Linux.
It provides abstraction for the construction of accelerated surfaces underlying Aura UI framework, input devices assignment and event handling.

Yet in 2013, Intel presented an Ozone/Wayland backend to the Chromium community. Although project progressed really well, after some time of active development, it got stalled: its most recent update is compatible with Chromium m49, from December/2015.

Leaping forward to 2016, a Wayland backend was added to Chromium upstream by Michael Forney. The implementation is not as feature complete as Intel’s (for example, drag n’ drop support is not implemented, and it only works with specific build configurations of Chrome), but it is definitively a good starting point.

In July, I experimented a bit this existing Wayland backend in Chromium. In summary, I took the existing wayland support upstream, and “ported” the minimal patches from Intel’s 01.org repository, being able to get ContentShell running with Wayland. However, given how the Wayland backend in Chromium upstream is designed and implemented, the GPU process failed to start, and HW rendering path was not functional.

weston_wayland_chromium

Communication with Google

Our main goal was to make Chrome capable of running natively on a Wayland-based environment, with hardware accelerated rendering enabled through the GPU process. After the investigation phase, the natural follow up step for us at Igalia was to contact Chrome hackers at Google, share our plans to Chromium/Wayland, and gather feedback. We reached out to @forney and @rjkroege, and once we heard back, we could understand why “porting” the missing bits from Intel’s original Ozone/Wayland implementation could not be the best choice on the long run.

Basically, the Ozone/Wayland backend present in Chromium upstream is designed in accordance to the original Mus+Ash Project’s architecture: the UI and GPU components would live in the same process. This way, the Wayland connection and other objects, that belong to the UI component, would be accessible to the GPU component and accelerated HW rendering could be implemented without any IPC.

This differs from Intel’s original implementation, where the GPU process owns the Wayland connections and objects, and the legacy Chromium IPC system is used to pass Wayland data from the GPU process to the Browser process.

The latest big Chromium refactor, since the Content API modularization, is the so called Mus+Ash (read "mustash") effort. In an over-summarized definition, it consists of factoring out specific parts of Chromium out of Chrome into "services", each featuring well defined interfaces for communication via Mojo IPC subsystem.

It was also pointed to us that eventually the UI and GPU components will be split into different processes. At that point, the same approach as the one done for the DRM/GBM Ozone backend was advised in order to implement communication between UI and GPU components.

Igalia improvements

Given that “ash” is a ChromeOS-oriented UI component, it seems logical to say that “mus+ash” project’s primarily focus is on the ChromeOS platform. Well, with that in mind, we thought we just needed to: (1) make a ChromeOS/Ozone/Wayland build of Chrome; (2) run ‘chrome –mash –ozone-platform=wayland’; (c) voilà, all should work.

WRONG!

When Fred and I started on this project two months ago, the situation was:

  • Chrome/Ozone/Wayland configuration was only build-able on ChromeOS and failed to run by default.
  • The Ozone project documentation was out of date (e.g. it referred to GYP and the obsolete Intel Ozone/Wayland code)
  • The Wayland code path was not tested by any buildbots, and could break without being noticed at this time.

After various code clean-ups and build fixes, we could finally launch Chrome off of a ChromeOS/Mash/Ozone/X11 build.

With further investigation, we verified that ChromeOS/Mash/Ozone/Wayland worked in m52 (when hardware accelerated rendering support was added to the Wayland backend), m53 and m54 checkouts, but not in m55. We bisected commits between m54 and m55, identified the regressions, and fixed them in issues 2387063002 and 2389053003. Finally, chrome –mash –ozone-platform=wayland was running with hardware accelerated rendering enabled on “off-device” ChromeOS builds.

screenshot-from-2016-11-14-00-51-02

 

Linux desktop builds

Our next goal at this point of the collaboration was to make it possible to build and run chrome –mash off of a regular Linux desktop build, through Ozone/Wayland backend.

After discussing about it on the ozone-dev mailing, we submitted and upstreamed various patches, converging to a point where a preliminary version of Chrome/mash/Ozone/Wayland was functional on a non-chromeos build, as shown below.

chrome_mash_ozone_wayland_non-chromeos-oct10

Note that this version of Chrome is stated above as “preliminary” because although it launches off of a regular desktop Linux build, it still runs with UI components from the ChromeOS UI environment, including a yellow bottom tray, and other items part of the so called “ash” Desktop environment.

Improving developers’ ecosystem

As mentioned, when we started on this collaboration, the Ozone project featured an out of date documentation, compilation and execution failures, and was mostly not integrated to Chromium’s continuous testing system (buildbots, etc). However, given that Ozone is the abstraction layer we would work on in order to get a Wayland backend up and running for Chrome, we found it sensible to improve this scenario.

By making use of Igalia’s community work excellence, many accomplishments were made
on this front, and now Ozone documentation is up-to-date.

Also, after having fixed Ozone/Wayland recurring build and execution failures, Fred and I realized the Ozone/Wayland codepath in Chromium needed to be exercised by a buildbot, so that its maintenance burden is reduced. Fred emailed ozone-dev about it and we started contributing to it, together with the Googler thomasanderson@. As a result,  Ozone/Wayland backend was made part of Chromium’s continuous integration testing system.

Also, Fred made a great post about the current Ozone/Wayland code in Chromium, its main classes and some experimentation done about adding Mojo capability to it. I consider it is a must-read follow on on this post.

Next steps

Today chrome —mash launches within an “ash” window, where the “ash” environment works as a shell for child internal windows, being Chrome one of them.
Being “mash” a shortener for “mus+ash”, our next step is to “eliminate”
the “ash” aspects from the execution path, resulting on a “Chrome Mus” launch.

This project is sponsored by Renesas Electronics …

renesas_logomark_l

… and has been performed by the great Igalian hacker Frédéric Wang and I on behalf of Igalia.

igalia-logo-364x130