Category: GNOME

Taking a quick look to the gnome-shell/mutter a11y

Randomly, when I have a litte time to, I’m trying to take a look about how the a11y support could be added to gnome-shell, and decide if Cally fits here.

So the first think to do is check the current status. After some stupid mistakes from my side (as trying to compile GnomeShell using the normal moduleset, instead of doing as the guide says), I was able to compile GnomeShell and get it running. Using the nested X server I got the next errors:

(mutter:24120): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup.

(mutter:24120): atk-bridge-WARNING **: IOR not set.

(mutter:24120): atk-bridge-WARNING **: Could not locate registry

This means that I was not able to load the atk-bridge using the nested server, so I used the replace option. Not a big issue right now, as gnome-shell is going really slow there, but I would like to use that on real development. Any quick ideas?

Ok, let’s go with the replace option, running Accerciser. What we get? Here a screenshoot:

Current status

I suppose that this gnomeshell-taskpanel came from gnome- shell ;), but we can see that just under mutter there isn’t any child. So, how to interact with this “Activities” button just on the top-left corner of the window?

Well, probably now I should have started to look at the mutter+gnome-shell code trying to analyze all their parts, but before that, I wanted to test a quick hack. Basically, see what happens if we just load Cally instead of GAIL. This can’t be done using the GTK_MODULES env (so don’t forget to clean it), as it is required to be done after init Clutter, and as there is no general way to do that on Clutter, this is required to made by hand. And by hand I mean modify directly Mutter, with a hackish patch.

Wow, wow. This is working, it doesn’t crash, both modules are loaded. Let’s test Accerciser. What we get? All the applications frozen. Hmm, well, this isn’t exactly what we wanted, but probably has sense. Probably Accerciser uses all the fancy AtkUtil methods related to event listeners, not implemented yet.

Well, no problem at all, I can return to the old times, and use dogtail. The good news is that now I’m able to see things under mutter application, and get their objects. For example the “Activities” button just in the top left corner. And click it! As easy as the screenshot shows:


Using Cally

Ok, so I’m writing on the python terminal, and getting the objects, and I click on “Activities”. What happens? That all are moved to another view (probally called activities view or something like that). See a example:


Activities View

Of course, I lost the focus on the terminal, so I can’t just type again “activities.click()” ir order to go back to the previous view. Since the POV of the automatic testing this is not a issue. For example, this little dogtail test works without problem. It clicks the activities button, and after two seconds, it click it again, so the view goes back to the normal one, no problem. But from the “classic a11y” POV this could be a issue. Although normally I don’t use AT apps like Orca, I always think that if I’m able to control a app just with the python terminal and dogtail stuff, any AT app could do that (this is not 100% true, I know). Here I found that this wouldn’t be as easy.

Anyway, this is not only about the AT apps. For example, in order to take the screenshot, I just clicked the “Impr Pant” so the save screenshot dialog appear. But of course, on one of the working areas. These working areas minimized at that moment, so to interact to that I needed to click on the working area and so on. IMHO, this is confusing, but after all, gnome-shell is just on development, but this leads if we need a app on the top of that view…

Well, and now the next steps should be do a real work instead of just change the module to load. About that, probably one of the other big issues is that we will need to manage a mixed environment GAIL-Cally, so I need to check how to do that. Other thing to review is just print all the things currently exposed (just mutter.dump()) and see if we miss something.

Clutter 1.0 released -> Cally 1.0 has born

Today Emmanuel Bassi finally announced the release of Clutter 1.0.0. Great news.

So, for some hours the Cally master didn’t compile against the clutter master. A new branch (cally-0.9) was created (although probably not required), and the master version was updated. As I explain on the README file at Cally, in order to make life easier, the version of Cally is tied to the version of Clutter, in order to avoid questions like “which version of Cally I need to install in order to support Clutter A.B.C?”. So, Cally 0.9.X will grant accessibility support for Clutter 0.9.Y. X and Y will kept independent, for internal usage, in a similar way to GAIL with Gtk+.

In the same way, I started to reduce the awful TODO file, and create some bugs on the clutter bugzilla, in order to trace the missing stuff and bugs in a sensible way. So, since today, Cally has several bugs.

Cally moved to Clutter repository and other news

As some of you already know, at first the Cally git repository was placed on Igalia server, waiting to be moved to the Clutter repositories. As one of the GCDS post-effects, this move was being made. So, the new way to get the Cally source code is:

  • git clone git://git.clutter-project.org/cally

And the development will continue there, so the Igalia one would be closed soon.

In the same way, a new component was added to the O-hand Clutter bugzilla, so now you can open a bug against Cally. Now I will start to transform the über TODO file at Cally in small bugs, in order to track the missing features and other bugs in a sensible way.

I would like to thank Emmanuel Bassi, Ross Burton and Richard Purdie for this move.

Other news:

  • For the people who went to my GCDS talk, I added a branch with the Clutter example I used. This is a heavily modified version of the clutter toy foofone called CrapPhone. This also includes a dogtail script. For the moment it will be maintained as a different branch, as I’m not sure if it is worth to include it with the rest of the examples on the master branch.
  • A first commit related to the AtkText support on ClutterText (CallyText) was pushed on the master, containing the implementation of most of the methods of this interface. It also adds an example (on the examples directory), that tries to use all this methods. This implementation is based on GailEntry, and the methods that still remains are the ones that on GailEntry was based on GailMisc. This library is really tied to Gtk, and some methods are not 100% useful on Cally. Probably a equivalent object (ie: CallyMisc) would be created in order to fulfill this functionality. In the same way, I think that worths to not create über commits, and split this a little. And probably I will start the support for AtkEditableText before finish that.

My last day in GCDS

Like a lot of other fellow igalians, I has been this week on the GCDS (Guadec+Akademy) at the beatiful city of Gran Canaria (beatiful and not as hot as I feared).

Probably I should have made more posts about that, but there were several talks to see, but more important, I used most of my free time to prepare my talk on the GUADEC. This task was scheduled yesterday, at 10:00. Yes, the first talk in the morning. And yes, the day after the sponsored Igalia party 😉 . But in the end there were more people than I expected (although in the same way, I started 5 minutes late by purpose 😛 ).

My talk was about Cally, the Clutter Accessibility Library implementation. In fact we can say that the Tuesday morning was the “Clutter morning”, as there were three talks about Clutter.

Except for the fact that in the middle of the talk the microphone fall to the floor, I think that it was fine. If you are interested, you can download the slides here.

As I said on the talk, currently Cally works, but for a specific scope. So, what about the future?

  • Complete the missing functionality (AtkText: work in progress)
  • Review new clutter-based toolkits (glitter, nbtk, …)
  • Review applications using clutter directly (gnome-shel, …)

Although the GCDS will finish on Friday, this is my last day here (I preferred to go back earlier) . As the song says, “all come to an end“, so I will come back to Coruña tomorrow.

I saw several interesting talks, I met some interesting people and I enjoy all the guadec, but in the same way I’m really tired right now 😉

Cally now with examples

The other day Alberto Ruiz reminded me that several functions on the clutter API have changed, so now it uses floats instead of integers.

Testing it, I notice that this lead to some compile errors on cally, but they were solved easily. However, I take this moment in order to add something more important. I realized that could be hard for anyone to to start to play with the library, as at this moment the cally module requires to be loaded by hand. This issue is documented on the TODO file, but I think that should be better to have some code showing how to do that.

In the same way, I think that for basic development process on cally, it is better to have clutter examples on purpose, instead of working on random existing clutter applications.

So, to solve that, I have added a directory in order to start to put the examples. At the moment it only have one, but at least I have added the infrastructure. The first example is really basic (a stage and tree rectangles), but I was able to detect a bug using it 😛

The compilation of the examples are deactivated by default, but you only need to execute the autogen.sh script adding “–with-examples”, in order to compile them.

CALLY: Clutter accessibility implementation library

During the first months of the year, Igalia has been working on the accessibility support for Clutter, funded by Nokia.

The last weeks we were trying to coordinate how to release it, so I would like to thanks Rodrigo Novo and Quim Gil, in this task.

In the same way, I would like to thank Emmanuel Bassi for the interest he showed in this library, although he is busy enough with the work for the imminent Clutter-1.0.

Using the same scheme for GAIL and HAIL, this library is called CAIL, Clutter Accessibility Implementation Library.
The library was is called CALLY, that is is similar to Clutter + a11y, but avoiding the numbers (ca11y was suggested by Juan A. Suarez, Emmanuel Bassi suggested use “l” instead of “1”).

At this moment the library is really in its beginnings. It is small, and there are a lot of things still to implement, but it works, and allow to expose the different Clutter objects to an AT application. But, it is this moment the best one to start to get feedback from other people, specially from gnome accessibility guys, when we have a working library, but not a huge amount of code really hard to understand as whole.

The first question that arises is if we are approaching this in the correct level of abstraction. Clutter is oriented to be a drawing technology, so in theory the ideal thing should be to implement the a11y support directly on any new clutter-based toolkit. In some way, this is similar to the Gtk+/Gdk relation, and implement a11y support on the Gdk level. It is worth pointing out that because at this level we are mostly missing functional information (objects to interact directly with, like a button), all objects are mostly informative.

But we don’t have news about an hypothetical fancy toolkit (tidy is just an example) and applications are using directly clutter, and probably for a long time, so maybe a direct a11y support will be required, and the work done here could be reused in the future for upcoming toolkits.

But, apart from this philosophical reflections about the existence of CALLY, there are some implementation details that are debatable too (so any feedback is welcome):

  • GtkAccessible equivalent not implemented: the idea was to implement CALLY without modifying Clutter at all. It is enough for the moment
  • Window system information is required to implement AtkComponent, so it depends on the backend
    • For the moment, it only supports the X11 backend
    • There is no implementation done to manage multiple backends
    • An option could be to add the backend support, like Clutter, but then, you need to be sure to install the correct flavor of CALLY with the correct flavor of Clutter.
    • Other option to ensure a synchronization between Clutter and CALLY could be to compile both together, and integrate CALLY as a Clutter module, like GAIL on Gtk.
    • After a fast review of SDL I didn’t find a way to implement AtkComponent properly. Is it viable on any backend?
  • Textures as buttons
    • ClutterActor has signals for “press” and “release”
    • Any actor could be used as a button; the most used one is the ClutterTexture
    • So the role should be different, ATK_ROLE_IMAGE==>ATK_ROLE_BUTTON, and the app should change that. The current experience is that they will not (or at least not easily).
    • Solution: change it on initialization if we detect that it has a handler bound to these signals.
  • AtkAction implemented on CallyActor
    • As ClutterActor has signals for “press” and “release”, it makes sense to implement AtkAction on CallyActor
    • So we should find a way to extend AtkAction implementation on subclasses
    • Used the solution implemented on GailCell
    • Maintain a list of action functions, and add a public API on CallyActor to add/remove that.
  • ClutterContainer
    • AtkObject uses a composite pattern. On GAIL GailContainer redefines these methods.
    • On Clutter ClutterContainer is an interface, so several Clutter object could implement it
    • Several AtkObject methods can be implemented in behalf of ClutterContainer.
    • So currently, the implementation of these methods on CallyActor is conditional. On runtime, it checks if the Clutter object implement this interface
  • The interaction with GAIL is not clear. Or with other words: the a11y support for applications using Clutter and Gtk+ is not clear yet
    • The existing a11y support for GtkClutterEmbed was not included, as was based on the assumption that the environment will be always using this widget
  • CALLY is using a module approach like GAIL.
    • Currently the applications are required to load this module by hand
    • With GAIL you load it by using GTK_MODULES=gail:atk_bridge
    • But there isn’t (AFAIK) CLUTTER_MODULES
    • Anyway it is not clear if this could be the best option, as GTK_MODULES is causing some problems in other environments

    An of course, a lot of missing functionality.

    You can read a detailed list on the TODO file I included in the source code.

    At this moment the code is in a public Igalia git repository. The idea is to move it someday to the clutter repository, but at this moment Emmanuele Bassi and other Clutter developers have a lot of work with the imminent Clutter-1.0 release, so we will try to improve CALLY meanwhile.

    So, if you are interested, you can get the code here:

    git clone http://git.igalia.com/cally.git

    Update: After the Ron Thompson comment, the library was renamed from cail to cally. Sorry for the nuisance.

    Maemo5 beta SDK is here

    Well, I’m not the first one to say that, as Quim, Joaquim, Berto, Claudio and Dape said that yet, but, hey!, the Maemo5 beta SDK is out ! 😛

    As it was said before, this Maemo5 change a lot the interaction with the user, as it is documented on the HIG Guidelines and on the Hildon Tutorial, oriented to interact with the screen directly, so now we have “touchable widgets” and panning effects. As a example, we have been working on the HildonTouchSelector widget, used to select a value, and in some way, the touchable equivalent to the GtkComboBox.

    Anyway, as Claudio and Berto announced, the big new is that now all the work is done publicly, and now there is a garage project page and a public repository.

    Enjoy! 😉

    More about GAIL

    In one of the old bugs related to GAIL you can find that:

    “In general we don’t want to export the other gail headers, since the implementation details of how ATK is provided for specific widgets is not something we want to require bincompat for, nor do we want these interfaces, in general, to be used externally (other than the ones in libgail-gnome).”

    This sounds pretty well. Anyway, this paragraph has a “but”. The reporter (Bill Haneman, one of GAIL’s fathers), opened the bug in order to make public the GailWidget header, as it is a real useful class.

    On one of my previous posts, I said that although you have an anonymous (run-time) inheritance from GAIL, this can’t solve the problem that without the header you can’t redefine the virtual methods of a GAIL object without reimplement it, or make something really hackish.

    Well, now I found another problem with this: GtkAdjustment. The a11y support on GtkAdjustment is implemented on the object GailAdjustment. But as you normally don’t require direct a11y support for this widget, this class is used as an aux class for classes like GailRange (GtkRange). Soooo, in the end, the current implementation of GAIL doesn’t define a concrete GtkAdjustment factory.

    Conclusion: if you want to use GailAdjustment outside Gtk+, you can’t do that using the “indirect subclassing method”, so if you want that, you need to reimplement it, or just C&P the code. And this is funny, because normally with the C&P anti-pattern you get two similar problems fixed with two similar solutions (the original code and his copy). In this case, with exactly the same problem, you get two similar (equal) solutions (the original and his copy).

    Random thoughts about a11y on clutter

    Clutter and a11y support are important issues related with Gnome, so it is normal that sometimes the question “a11y and Clutter” arise. Some time ago this appears on the desktop devel list, and Emmanuel Bassi closes says that there are a plan for it in the future [1]

    And now, this topic has return, this time on the clutter mailing list [2] [3]

    Some time ago I was playing with this two topics, and I made some coding tests too. As this codings tests are still too basic, here [4] is a link to a pdf with my first thoughts after this implementation, so anyone could read about it if he wants.