{"id":754,"date":"2017-06-16T11:01:46","date_gmt":"2017-06-16T09:01:46","guid":{"rendered":"http:\/\/blogs.igalia.com\/jaragunde\/?p=754"},"modified":"2022-10-03T16:14:05","modified_gmt":"2022-10-03T14:14:05","slug":"genivi-fying-chromium-part-3-multi-seat","status":"publish","type":"post","link":"https:\/\/blogs.igalia.com\/jaragunde\/2017\/06\/genivi-fying-chromium-part-3-multi-seat\/","title":{"rendered":"GENIVI-fying Chromium, part 3: multi-seat"},"content":{"rendered":"

In the previous blog posts<\/a>, we described the work to bring the Chromium browser to the GENIVI Development Platform<\/a> (GDP) using the latest version of the Ozone-Wayland<\/a> project. We also introduced our intention to develop multi-seat capabilities on that version of the Chromium browser. This post covers the details of the multi-seat implementation.<\/p>\n

Goal<\/h3>\n

The GENIVI stack is supposed to allow applications run in multi-seat mode. A seat is a set of input\/output devices like, for example, a touchscreen and a keyboard; one computer (the head unit) connected to several seats should be able to assing applications to each seat and let them run independently. Hence, our goal is to let one Chromium instance manage several browser windows at the same time and independently, getting their input from different seats.<\/p>\n

\"\"<\/a>

Renesas Salvator-X board running Chromium on two seats<\/p><\/div>\n

<\/p>\n

Problem<\/h3>\n

We started with an analysis of the browser on a multi-seat environment, comparing its behavior with other applications, and we identified some problems. In first place, we noticed that keyboard focus could be stolen by other browser windows; in second place, we found that only one browser window was receiving all input events regardless of seat configuration.<\/p>\n

Let me first illustrate the flow of events between Chromium process in Ozone-Wayland:<\/p>\n

\"\"<\/a><\/p>\n

All browser window surfaces belong to the GPU process, events that affect those surfaces arrive to this process and then they are sent to the browser process via internal IPC. There, events would be processed and their effects sent to the render processes if necessary.<\/p>\n

The concept of “focus”, as implemented in Ozone-Wayland, means there can only be one focused window, and that window would receive all kinds of events. All events that are received by the GPU process that belonged to different surfaces\/windows are merged and received by the focused window in the browser process. Important information is lost in the process, like the original device ids. Besides, there is no awareness of the different seats in the browser process, and the GPU process ignores that info despite having it.<\/p>\n

Solution<\/h3>\n

The basis of the solution is to break the assumption of having only one focused window and integrate seat information in the event flow.<\/p>\n

We started by creating separate concepts of keyboard and pointer focus, which fixed the first issue for the most part. For the complete solution, we also had to add extra wires to link seats and devices in the GPU process using already existing information, and transfer the required information to the browser process. In particular, we added extra information to the internal IPC messages related to the device ids that produce every event. We also added the concept of seats in the browser process, with new IPC signals to sync the seat objects and seat assignment information. This information is obtained using the ivi-input interface from the Wayland IVI Extension project<\/a>.<\/p>\n

You can see a class diagram with the highlighted changes (blue: added, red: removed) below:<\/p>\n

\"\"<\/a><\/p>\n

The multi-seat implementation of Ozone-Wayland described above is available in my GitHub fork<\/a>, branch wip\/multi-seat<\/a>.<\/p>\n

Testing it<\/h3>\n

Patches haven’t been merged yet into genivi-dev-platform<\/a> master, but there is a chromium branch<\/a> with all the integration work so far. The last PR has been recently merged<\/a>, which includes multi-seat, patches to support the Salvator-X board and a backported fix for the Wayland IVI Extensions.<\/p>\n

You can already do your own builds by cloning the genivi-dev-platform chromium branch<\/a>. Then, just follow the GDP instructions<\/a>. We have successfully tested the multi-seat implementation on Intel hardware and also on Renesas R-Car generation 3 boards like the Salvator-X shown above.<\/p>\n

If you are building your own HMI controller, you have to use the Wayland IVI Extension APIs<\/a> to properly setup the screens, layers, surfaces and seat assignment. Seat configuration is done via udev<\/a> (see Advanced use<\/em>). For testing purposes, you may want to use the LayerManagerControl<\/code> command-line tool to simulate the HMI controller; I can share with you the commands I used to setup the Salvator-X with two seats: two keyboards and two touchscreens, one of them plugged via VGA and another one through HDMI.<\/p>\n

In first place, this is my udev configuration to create the seats, in the file \/etc\/udev\/rules.d\/seats.rules<\/code>. Touchscreens are identified with their physical USB address because they are the same brand and model:<\/p>\n

ATTRS{name}==\"Dell Dell USB Keyboard\", ENV{WL_SEAT}=\"seat_1\"\nATTRS{name}==\"Logitech USB Keyboard\", ENV{WL_SEAT}=\"seat_2\"\n\nATTRS{phys}==\"usb-ee0a0100.usb-1.1\/input0\", ENV{WL_SEAT}=\"seat_1\", ENV{WL_OUTPUT}=\"VGA-1\"\nATTRS{phys}==\"usb-ee0a0100.usb-1.2\/input0\", ENV{WL_SEAT}=\"seat_2\", ENV{WL_OUTPUT}=\"HDMI-A-1\"\n<\/code><\/pre>\n
To manage layers, surfaces and focus on my own, I had to stop the GENIVI HMI:<\/p>\n
systemctl --user stop gdp-new-hmi\n<\/code><\/pre>\n
I started by setting up one layer for each screen, with sizes that match the screen resolutions:<\/p>\n
LayerManagerControl create layer 1000 1024 768\nLayerManagerControl set layer 1000 visibility 1\nLayerManagerControl set screen 0 render order 1000\n\nLayerManagerControl create layer 2000 1280 720\nLayerManagerControl set layer 2000 visibility 1\nLayerManagerControl set screen 1 render order 2000\n<\/code><\/pre>\n
Then I ran the chromium browser (you will probably want to have several terminals open into the device for convenience), I ran the command twice to get two browser windows with surfaces 7001 and 7002. I configured the surface sizes and assigned them to each layer:<\/p>\n
LayerManagerControl set surface 7001 visibility 1\nLayerManagerControl set surface 7001 source region 0 0 1728 1080\nLayerManagerControl set surface 7001 destination region 0 0 1024 768\n\nLayerManagerControl set surface 7002 visibility 1\nLayerManagerControl set surface 7002 source region 0 0 1728 1080\nLayerManagerControl set surface 7002 destination region 0 0 1280 720\n\nLayerManagerControl set layer 1000 render order 7001\nLayerManagerControl set layer 2000 render order 7002\n<\/code><\/pre>\n
Finally, configured seat acceptances for each surface to receive events from only one seat, and gave keyboard focus to both:<\/p>\n
LayerManagerControl set surface 7001 input acceptance to seat_1\nLayerManagerControl set surface 7002 input acceptance to seat_2\nLayerManagerControl set surfaces 7001,7002 input focus keyboard\n<\/code><\/pre>\n
This work is performed by Igalia<\/a> and has been made possible by the funding provided by the GENIVI Alliance<\/a> through the Challenge Grant Program. Thank you!<\/p>\n
\"GENIVI<\/p>\n","protected":false},"excerpt":{"rendered":"
In the previous blog posts, we described the work to bring the Chromium browser to the GENIVI Development Platform (GDP) using the latest version of the Ozone-Wayland project. We also introduced our intention to develop multi-seat capabilities on that version … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22,23,24,3],"tags":[26,25],"class_list":["post-754","post","type-post","status-publish","format-standard","hentry","category-automotive","category-browsers","category-chromium","category-igalia","tag-chromium-on-genivi","tag-genivi"],"_links":{"self":[{"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/posts\/754","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/comments?post=754"}],"version-history":[{"count":18,"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/posts\/754\/revisions"}],"predecessor-version":[{"id":1127,"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/posts\/754\/revisions\/1127"}],"wp:attachment":[{"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/media?parent=754"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/categories?post=754"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.igalia.com\/jaragunde\/wp-json\/wp\/v2\/tags?post=754"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}