23
Nov 21

Drop shadows on Linux, or why standards are good

Since the origins of graphical desktop environments, there were two approaches to styling GUI of an application: using the standard system toolkit versus choosing a custom one.

When a single platform is targeted, choosing the approach is often the matter of aesthetics or some particular features that may be supported in certain toolkits. The additional cost of adopting a custom toolkit may actually be a one-time investment, and if the decision to use it is taken at the right time, the cost may be low. However, when it comes to cross-platform applications, using a cross-platform toolkit is the obvious choice.

GUI toolkits do a good job at rendering the contents of the window, but there is an area where they usually step aside: window decorations. Even if we look at cross-platform toolkits, the best they can do is provide some façade for the standard options available on supported platforms. But what if we want to customise everything?

Let us take a look at some random window in a modern desktop environment.

This is KCalc, the standard calculator application built into KDE Plasma desktop environment.

KCalc, the standard application built into KDE Plasma

What if we wanted to replicate that on our own? At first glance, no big deal. Drawing the title bar would not be that difficult, as long as we render everything in the window. The border is easy too, and rounded corners are also feasible if the window manager supports transparency.

But the window also has a drop shadow. We have to render it too, and this is where things become tricky.

KCalc vs. Chromium.  Drop shadows look quite different.

KCalc vs. Chromium, note how different the shadows are

Yes, the drop shadow is essentially just one more area inside the window, we have to render it, and also we have to make things around it work smoothly. The inner strip of the shadow should work as the frame of the window where the user would see the resize mouse pointer (and it should work that way), while the outer part should be totally transparent for the mouse events, but not totally—to the user’s eye.

The outermost rectangle is the edge of the “real” window; the innermost one is the “logical” one. The narrow strip (partially striped) that borders the logical window is the resize area.

Basically, to be able to do what we have just explained, we need two things. The first one is support for transparency in the window manager. The second one is some way to tell the window manager where our “logical” window resides within the “real” one, so that the environment could correctly snap our window to the edge of the screen or to other windows when we drag it there. (The inner part that makes sense as a window to the user is often called “window geometry”.)

On Wayland, transparency is always supported (yay!), and the concept of the window geometry is part of the desktop shell protocol, such as xdg_wm_base. Both requirements are met.

On X11 it is more complicated. First, transparency is not always supported, but let us assume that we have that support, otherwise we cannot have any shadows. The major pain is setting the window geometry, or to say better, the lack (at the moment of writing) of a standard way to do so. There is a _GTK_FRAME_EXTENTS window property that, as its name suggests, was once introduced in GTK. There it seems to be used to define margins at the edges of the window—you may ask, “it seems”? Are you not certain? Well, yes, because that property is not documented. There are a few other posts about this issue on the internet. I would recommend What are _GTK_FRAME_EXTENTS and how does Gnome Window Sizing work? by Erwin and CSD support in KWin by Vlad Zahorodnii.

Currently _GTK_FRAME_EXTENTS is supported by GNOME (naturally) and KDE Plasma (reverse engineered). In other desktop environments (or to say better, in window managers other than Mutter and KWin) setting it may cause weird issues.

Precisely that issue is what happened to Chromium.

In regards to the window decorations, the Linux port of Chromium was a bit backwards for a very long time. It had an old style thick frame with sharp corners and without the drop shadow. Finally, that had been improved, and the modern window decorations were shipped in Chromium version 94. The new implementation used _GTK_FRAME_EXTENTS to define the shadow area.

Soon after that, a bug report came from users of Enlightenment. In that environment things inside the Chromium window went mad, mouse clicks strayed from the actual position of the pointer. The quick investigation (it was really quick thanks to the help of people who reported the problem) showed that the culprit was that very window property. The window manager got confused when the frame extents were set to zeros for a maximised window, instead it expected the property to be reset completely.

Soon after we landed the fix, and people from Enlightenment confirmed that the issue was resolved, another bug report came, this time from Xfce. There, the investigation was a bit longer, but finally we found (thanks to the help of people who reported the problem and to the maintainers of the window manager) that the window manager in that environment actually expects quite the opposite: for the maximised window it wants all zeros, and gets confused if the property is reset completely.

The situation came to a dead end. Two window managers wanted exactly the opposite things. What could be done to resolve the issue? We could easily end up having workarounds for every non-standard window manager, which is one of the most unpleasant situations in software maintenance.

Luckily, the maintainers of Xfwm4 (the window manager in Xfce) suggested fixing the issue from their side, and landed the fix really promptly. So this story has a happy end!

Or rather, the story will have a happy end, because we still had to put in a workaround for Xfwm4 that disables window decorations on that window manager. The workaround is temporary, and we will remove it once Linux distributions that base on Xfwm4 adopt the fix.