Freie Freude

Just a quick (and slightly delayed) note to let you all know that after seven years I have left Igalia. It has really been an amazing, rocking place to be, and I have the best wishes for the company and its people.

Malmö seaside view, with the bridge to Denmark in the background.

Currently, I am living in Malmö (Sweden), where I am taking part in the Master in Interaction Design. This will be my second Master’s degree, after the one on HCI that I studied at the University of York in 2009. But whereas that was more focused on theory and academical research, this one is eminently practical: the course is structured around projects that try to go beyond the boundaries of mainstream everyday design, using many different techniques and tools to gather insights and quickly build working prototypes.

I don’t know yet what the future holds, but that is a question for the new year.

This blog will be frozen after this post; if you want to keep in touch, I’m  @felipeerias on Twitter.

Some time ago, I wrote a small functional prototype to explore some of the design ideas for the evolution of the GNOME Web browser (maintained by my colleagues at Igalia). I thought that it would be a good idea to show these experiments to a wider public.

The basic idea by the GNOME designers is that, instead of tabs, open pages would be placed in an overview: you would click on a thumbnail there to return to a certain web page, and clicking again on “Pages” would take you back to the overview. A possible evolution of this would be to integrate bookmarks and reading lists in that overview.

This first video shows the interaction as described in the current design: in the overview, open pages are shown in a horizontal list, which gets reordered so that the leftmost element in the list corresponds to the last open tab. Note how the thumbnail is updated whenever we go back to “Pages”, and how the list scrolls to the left to show the most recently opened sites.

(link to video)

I also implemented an alternative UI where the open pages are arranged in a static 2D grid. Here it is:

(link to video)

This little application was written in a bit over 200 lines of QML. The code is available here:

• http://people.igalia.com/femorandeira/files/Ephy_20120530.tar.gz

The project folder includes compiled binaries that should work on, at least, 64-bit Debian and Ubuntu. Just uncompress it and run

cd Ephy ; ./Ephy

Note that if you want to build it yourself, you will need the qt4, qt-webkit and qmlviewer development libraries for your distribution; then, you can just run

make distclean ; qmake && make

This is a small summary of my talk at GUADEC 2012 in A Coruña, Spain.

I got the talk started with a quote from Bill Buxton‘s book “Sketching User Experiences”:

“Getting the design right, and the right design”

“Getting the design right” refers to the things that concern us when refining a design: usability, accessibility, visual appearance, performance, etc.

However, what exactly is “the right design”?

Here are photos of three objects to help us start thinking about this.

“Cars in Cuba - 57”, by patrick_nouhailler

The first one is an old car in Cuba, carefully kept and maintained. Terribly attractive at a first look. After this first impression, questions begin to come up. Is it easy to handle? Expensive to maintain? Ecological?

“My Armory: Chef's, Boning, Utility Knives”, by osakajon

The second photo shows a set of Japanese knives. Are they beautiful? Kind of, as far as knives go. They feel great in the hand, though, and are sharp and sturdy. And what do they tell about their owner? What would it say about yourself to own a set like this?

“La petite tour”, by esm723

The third photo is a small figurine of the Eiffel Tower. It is not particularly attractive. It is absolutely, perfectly useless. And yet, it has value for its owner. But that value does not come from its looks of its usage. The value of this object is not in the object itself: it comes from how it makes its owner think (in this case, of a past trip to Paris).

There have been different attempts at defining the characteristics that make things attractive and valuable for us. In his book “Emotional Design”, Donald Norman details three levels of processing:

• Visceral: related to the sensory properties of a product
• Behavioural: how it feels when used
• Reflective: how it makes us think differently about ourselves, and how it changes the perception of others

A more recent approach has been carried out by Karen Holtzblatt (InContext). She and her team identified a set of characteristics that make a product “cool”:

• Allows you to accomplish your intent anywhere, on your time
• Goes direct into action without hassle
• Provides connection with the people that matter to you
• Helps you build your own identity
• Has nice aesthetics and sensation

Laseau's funnel

Laseau’s funnel describes design as a combination of ideation and reduction. There are many different solutions to explore at the beginning of a project, and we need to be able to go through all those ideas quickly, evaluating and rejecting them. Making questions upon questions in order to start getting the right answers.

Requirements-driven development is not enough to provide the characteristics that people really value. A much better approach is to be able to quickly try out design ideas, using those sketches to help us elicit the real user requirements and refine the designs.

These sketches need to be fast and cheap, so that they can be plentiful. And because they are plentiful and cheap and fast, they can be disposed of, to leave room for more and better ideas.

As it is often said, design is about saying “no”: design is a negative craft.

Besides its benefits for ideation, this approach also helps improving the communication of design ideas among designers, and  between them and other stakeholders (i.e. the GNOME community at large, in our case).

The Wizard of Oz

One (surprisingly) influential film for the field of UX design has been “The Wizard of Oz”. You are probably familiar with the story: Dorothy’s house is carried away by a tornado, all of a sudden she is not in Kansas any more, she meets a cast of wacky characters and runs into several adventures until she and her friends finally meet the famous Wizard of Oz.

Large, loud and surrounded by fire, the Wizard is a terrible and dreadful sight. Dorothy and her friends are appropriately afraid and have no choice but to do as he commands. Until the little dog Toto gets so scared that he runs and pulls at a curtain: behind it there is a small old man, pulling levers.

So the terrible Wizard was actually just an old man using smoke and mirrors. But what’s interesting is that, up until the point of this revelation, Dorothy’s and her friend’s reactions were governed by the belief that he was real. The Wizard was fake, but their experiences of him weren’t.

The Wizard of Oz is actually the name of a specific technique for sketching interactions where part of the behaviour of a computer system that would be too expensive to build is replaced by a hidden human operator. It has been successfully used to, for instance, simulate the experience of using a perfectly accurate speech-to-text system way before those systems were as common as they are beginning to get today.

Palm Pilot wooden prototype (1995)

The talk continued with descriptions of a number of examples and different techniques to quickly sketch the behaviour and experience of using a product way before that product is a reality. Several tangible prototypes were discussed, from simple paper-based sketches to full living rooms.

Living room at CWI Amsterdam, used to conduct experiments on different TV and teleconferencing technologies.

Some of the examples relied on storytelling engage the audience’s empathy to help them experience the proposed interaction through the protagonists of the story. The cases discussed ranged from written stories to the acting up of interaction flows and even professional theatre performances.

Theatre used to elicit requirements from elderly users, University of Dundee

In GNOME, we often use wireframes and visual mockups. A couple of interactive examples that were discussed in the talk were:

• This demo of multiple selection by Jakub Steiner (link)
• And my own experiments with the navigation on Epiphany (code, videos)

Communication of design decisions is a problem in GNOME. We are a project composed of small teams working on different and remote organisations, but with a lot of potential stakeholders (a whole community of them!). It is important to develop and practice techniques that would allow us to refine and communicate ideas more fluently.

To end with a bit of advice: just turn your computer off from time to time, grab some pen & paper, and try things out 🙂

Images:

• https://secure.flickr.com/photos/patrick_nouhailler/5507717586/
• https://secure.flickr.com/photos/osakajon/127131454/
• https://secure.flickr.com/photos/esm723/3074137958/
• http://www.imdb.com/title/tt0032138/
• http://www.computerhistory.org/collections/accession/102716262
• http://www.cwi.nl/
• http://www.computing.dundee.ac.uk/staff/afn/

My blog has just been added to Planet GNOME, so let this post be my way to say that I am very happy to join the fine people there (including several colleagues at Igalia).

This week I will be attending the CHI2012 conference in Austin, TX.  This is the main conference on human-computer interaction and UX design. I will write a long recap once it is over, with special attention to those bits that may be interesting to GNOME and other work that we are carrying out at Igalia, and also to things that are just too wonderful, eye-catching or plain weird to miss.

Following up on my last post, I want to share a few ideas that could improve the use of the Web from GNOME. Many of these come from other people and I am trying to combine them into one coherent package.

The first goal would be to offer better support for common Web browsing patterns, revisitation and exploration. Specifically, this means supporting web applications, a more convenient and agile presentation for favourites, better history and bookmarks management, better tab management within the browser window for pages that are related to the same tasks, and better tab management from the Shell to help the user align the different sets of tabs with his current activities and interests.

The second goal would be to do so in a way that is not cumbersome and complex, but light and consistent.

Web apps

Most of the next million apps written will be web applications. The browser should acknowledge this, allowing the user to turn a Web site into an application that can be accessed like any other.

Launcher for a GMail app.

Revisitation: Home and History

As noted in my previous post, there are different kinds of Web revisitation; one of them comprises sites that we visit often because they lead to new information, which is not exactly the same as storing a linking to a page because of the information that it contains at the moment of reading (e.g. an article). In a manner similar to what Firefox does, I propose to have a Home tab as the starting point for browsing. This tab could include a search field, links to recent pages and groups of pages, favourites and Reading List. Being able to define a page as “favourite” and “pin” it to the Home page would ease mid- and long-term revisitation, which makes up for a large percentage of our activity in the Web.

The Home tab would be a way to get to new content, but what about returning to sites that were visited some time ago? Next to the Home tab, we could place a History&Bookmarks tab that offered a rich search interface to retrieve pages that have already been seen.

Tabs on top, with Home and History on the top left.

Fine-grain tab management

Modern browsers are placing their tabs on top with good reason. The main advantage is that this helps establish a visual hierarchy inside the browser window that reinforces the proper mental model, so that controls that operate on the same scope are grouped together. To decide which controls should be given priority in the interface, we could use usage data from Firefox as a guide, always keeping in mind that we cannot assume that everybody will know how to use all the available shortcuts (e.g. a similar study found that over 80% of users never used Ctrl+F to search). Browser-level functionality (New Window, Preferences, Quit…) could be moved to the application menu.

Tabs provide a number of benefits that make them a convenient way to organise your Web browsing. However, one of their problems is that as their number grows, it can become difficult to go back to a certain tab; a way to improve this situation could be to show a thumbnail of the tab’s content on mouse-over, allowing for a quick scan of open tabs without having to open them one by one.

Tab thumbnail on mouse-over.

There is a difference between following a link and opening it in a new tab. In the first case, the original page is still visible and readily accessible; in the second, it has disappeared from the UI and has to be kept in the user’s memory, to be accessed again via the Back button. These two different actions can allow the user to create a curated version of their trail through the Web, one that does not contain all the pages that they have visited but just those that have been deemed important. These tab trees are an important feature but tab-focused interfaces (e.g. tree-style tabs, other ideas) might be far too complex. A compromise could be to include a visual hint at the existence of different tab groups, but without making it the main point of the interface.

Without text, can you tell which of these tabs are related?

Coarse-grain tab management

Tabs are a good way of structuring your browsing when their number is low enough (research shows that an usual number of open tabs is around 6). When their number grows, you can have trouble because there are simply too many unrelated tabs in one window. So we have a problem with the organisation of a lot of content that is related to different activities: well, the GNOME Shell is a solution for that. I propose to allow high-level management of Web tabs directly from the Shell Overview (not too different from Panorama with a bit of this), providing an overview of the open tabs and supporting their movement between different browser windows and workspaces.

Epiphany window in the Shell overview, displaying the open tabs.

Wrap-up

I have tried to describe a situation where Web browsing is more tightly integrated with the desktop. There is still a lot of work to do: detailed functionality needs to be refined, assumptions need to be verified, mockups and prototypes need to be created and evaluated…

A browser is a very complex application to design, but luckily there is a lot of knowledge already available that should help us generate ideas and make informed decisions.

Earlier this week I began to look at some of the many available works on the field of Web browsers for the desktop, with the goal of improving the design of the Epiphany browser and taking advantage of the fact that Igalia is one of the main maintainers of WebKitGTK+. The first task, of course, is to correctly understand the problem: in a field as big and complex as this, this means a lot of reading and synthesising. Today I will explore two particular aspects: revisitation and tabbed browsing. In the future I will expand on this and begin to share some design ideas.

Revisitation

Revisitation means to access web sites that have been already seen previously. Although there are discrepancies on how to measure it, for the sake of design we can say that we have already seen roughly half of the pages that we visit. The article by Obendorf et al. mentions three kinds of revisitation:

• short-term revisits (within the hour): these are the most common, often performed by following links, or using the Back button;
• mid-term revisits (within the day): the most usual way is to use bookmarks or write the URL (often helped by autocomplete);
• long-term revisits: this is related to the rediscovery of information that has already been seen; people re-access these pages mainly through links because they need to re-search (enter the same search terms) and/or re-trace (follow the same steps); history and bookmarks are also employed to some extent, but the current interfaces might not be easy or convenient to use.

Previously-unseen pages are usually visited by directly entering a URL or by following links from search pages (e.g. Google) or other information hubs (e.g. reddit, news sites).

A wider research was carried out by Adar, Teevan and Dumais. Their findings are consistent to those above, as they found that Web page revisitations could be clustered in the same three groups plus another one, which they called hybrid and which contained sites that were popular but infrequently used. They went further in trying to analyse the kind of web sites that typically fell on each group. The fast revisitation pattern often corresponded to “hub&spoke” behaviour, where users move back and forth between a set of promising results and each individual item. The mid-term one tended to refer to pages that act as starting points where the user can carry out a task (e.g. communication, banking) or access new information (e.g. news, forums). The infrequently-accessed group comprised pages that provide specialised search (e.g. travel) or related to weekend activities; as in the previous paper, the researchers also note that external search engines are often used for revisitation. There was a fourth, hybrid group of pages that caused “hub&spoke” movement but that were infrequently accessed, such as craigslist, eBay, shopping, games, etc.

With these results, the researches mention a number of implications for design. The most interesting for me is that “there may be value in providing awareness of, and grouping by, a broader range of revisitation patterns. For example, users may want to quickly sort previously visited pages into groups corresponding to a working stack (recently accessed fast pages), a frequent stack (medium and hybrid pages), and a searchable stack (slow pages).”

Research on tabs

During the last years the usage of tabs has made the Back button less prevalent. For instance, a common behaviour is to perform a search and then open different results in their own tabs, attempting to find the desired information through exploration of the candidates without losing track of the result set for further refinement. This often causes problems because the Back button does not work as expected (local history only applies to the current tab) and it might be complicated to find the originating document in the case of large tab trees. Problems with the Back button also arise when entering information through web forms and when using web applications.

A study of tab usage on Firefox showed that tabs are mostly used for immediate revisitation and task-switching. They serve as reminders or short-term bookmarks, they allow users to open links in the background and are a convenient way to keep frequently-accessed pages open. Visually, they are cleaner, less cluttered and easier to access that separate browser windows.

Many participants used tabs for revisitation more often than the back button, up to the point where, for frequent tabs users, tab switching was the second most frequent thing they did in the browser (after following links). The reasons reported were that tabs were more efficient, more convenient and more predictable (you can see the target right away). Another factor that helps to ease multitasking is that tabs leave the page in the same state, which is not always true with the back button.

The study found marked differences between regular and power tab users. The median number of open tabs was reported at around 6, but the maximum number of open tabs at one point in time could get much higher than that, past 20 and beyond for some users. As the participants were using regular Firefox, it could be that for some a limiting factor to the number of open tabs might simply be lack of space.

A bit of personal experience

As a user of the Tree-style Tabs extension for Firefox, I often find myself creating long trees of tabs where the tree itself marks a trail that is coherent and useful. I do not use the Back button often, and I think that the reason might be that opening a new branch in the tree somehow makes that part of the trail useful, clear and important, whereas pages that can only be accessed by going back soon fade out of memory. For a given task, it might well be that there is value in having a clear structure of related sites: the combination of tree-style tabs and the Back button helps create and navigate this structure.

Opening a link in a new tab actually marks the previous one as interesting and worth keeping around for a while, whereas closing a tab or following a link signals that the previous page was not that interesting after all (and it will fade from memory soon). This way of looking for information is probably related to orienteering, an information seeking strategy in which users take small steps towards their target using partial information and contextual knowledge as a guide. Making said set of steps visible and semi-permanent also acts as a very convenient reminder: my tab structure is kept between sessions, which makes it very easy to resume work or reading (for instance, there is a small subtree hanging from my RSS reader tab with articles that I will read later, and another one hanging from Bugzilla with bugs that I am working on).

Longer-term revisits

Regarding mid- and long-term revisits, I propose to contemplate three kinds of sites: web applications, information hubs and the personal archive. Web applications are self-contained and focused mainly on one task; their goal is in most cases to replace local applications for e.g. email, calendar, project planning, music, etc.

It might make sense to separate frequently visited websites that periodically provide new content from concrete and interesting information items; you can think about as the difference between reading the newspaper everyday and cutting out a news item that mentions your amateur football team. Information hubs are pages that are visited often because they lead to the discovery of new information, either on the same site (e.g. guardian.co.uk) or on others (e.g. reddit.com). On the other hand, the personal archive is a collection of information items that are relevant for the user because of the information that they already contain. There are many motivations behind the construction of personal archives: not just simply storing things for later retrieval, but also creating a legacy, making it easier to share resources, reducing fear of loss, self-expression and self-identity.

References

“Web Page Revisitation Revisited: Implications of a Long-term Click-stream Study of Browser Usage”, Obendorf et al., CHI 2007 Proceedings [PDF]

“A Study of Tabbed Browsing Among Mozilla Firefox Users”, Dubroy et al., CHI 2010 [PDF] [Presentation]

“Large Scale Analysis of Web Revisitation Patterns”, Adar et al., CHI 2008 [PDF]

“To have and to hold: exploring the personal archive”, Kaye et al., CHI 2006 [PDF]

“The Perfect Search Engine Is Not Enough:A Study of Orienteering Behavior in Directed Search”, Teevan et al., CHI 2004 [PDF]

Alex Faabor’s blog

Mozilla’s blog of metrics

Design proposals for Epiphany: EpiphanyRedux, hbon’s mockups

Thanks to Igalia, I am in Berlin this weekend enjoying the Desktop Summit!

Today I have been reading about the work done by the Distributed Computing group at the Zurich Institute of Technology. These guys are developing complex techniques for combining “audio social information” (tags and listening habits, taken from Audioscrobbler) to discover relationships, thus generating a social audio space where songs/artists are placed and that the user can navigate. An opposite (or maybe complementary?) approach to this one would be the use of audio features to establish relationships between songs/artists, but this might ignore some of the semantic and social information that is so relevant to our understanding of music. They have developed a set of prototypes for Android (with video) and Amarok. The science looks interesting and these kind of approaches could well be the next step in the evolution of music players now that the average collection is often too big to be dealt with using only long text lists.

Ok, back to the paper now (Social Audio Features for Advanced Music Retrieval Interfaces, if you are interested).

Today I want to think about a possible mobile application to fulfil the goal of getting “a varied stream of my music that suits a general theme and that requires little interaction from my side”. For the particular case of mobile music:

• Music can be in the background, allowing the user’s attention to be directed elsewhere.
• Portable music devices are often pocketed away. Direct interaction and user input are therefore sparse.
• Mobile music has usually only one listener.
• Fine management of the specific songs being played and their sequence might not be required.

These are all assumptions, but I will go with them for now.  Bear in mind that this is just a bit of exploratory design and so far hasn’t been validated.

The main idea is that the user should be able to just select the musical “theme” and get music along it. Since both tags and genres have a subjective component, I thought that using music artists as the “seed” for the radio stream might work well. If this sounds similar to Last.fm‘s radios, yes, it is similar.

Last.fm uses streamed content and relies on crowdsourcing to generate the relationships between artists, tags, songs, etc… One of the constraints that I had in mind when I thought about this application was that the music should come from the user’s personal collection and not depend on having an Internet connection. More importantly, Last.fm’s radios offer a wide range of choices and therefore require complex UIs, ; one of the questions that I am wondering here is if one could strike a compromise between providing music that satisfies the user’s needs and doing so with an interface that is simple, quick and pleasant to use.

This is a quick sketch. In this hypothetical app., the user is presented with a list of the artists and bands in their collection.

An image of the band could be used to make the application more visually attractive and easier to recognize. It is worth noting that the list layout shown in the image is not necessarily the best one for this particular case. The proper thing to do if we were serious about developing this would be to prototype, test and evaluate different possible list layouts in a systematic way: one column, two columns, images, no images… This might make a good starting point for a future post about testing.

It is worth repeating that this is supposed to work on the user’s music collection on a mobile device, which on average [citation needed] would be limited to dozens/hundreds of artists, most of which the user can recall or at least recognize [citation needed]. A different problem altogether would be to explore a vast collection that has been compiled by somebody else, as would be the case with e.g. Jamendo.

When the user selects one of the items, the application begins to play a stream of music that is related to the artist selected. The definition and calculation of similarity in music is a complex engineering topic that I will cover in future posts. Determining the “suitability” of the algorithm selected would not be trivial since what we are looking for is to provide the selection of music that better fits subjective desires and expectations, which change from person to person.

The “now playing” screen is intentionally simple: you can pause the current song, skip to the next one or go back to the list in order to select a new band. I am not even fully convinced that the “settings” item makes sense there, but it could act as a place to keep functionality that does not need to be always present (e.g. play only music from the selected band and not from related artists).

That’s all for today. Stay tuned for more stuff coming in the next days!

The first problem that I want to talk about in this new version of the blog is that of managing, browsing and consuming media content, with an initial focus on music and mobile devices. A good summary of current technologies and research can be found in Tim Langer’s paper “Music Information Retrieval & Visualization”.

Thinking about the systems’ user interfaces, we could imagine a graph displaying the level of control about the concrete songs being played versus the attention and frequency of interaction required (before and during playback). Different ways of enjoying music could be displayed in this graph, for instance:

• at one corner we would find the traditional radio, which requires no interaction except for selecting the station and offers no control of the music being played;
• Last.fm radios would be nearby: they require a more careful selection and offer a bit more control during playback (skipping songs);
• on the other end of the graph, we could think of a music game where the user can not select the music being played but is required continuous interaction to win (the interaction being the whole point of the system);
• the extreme combination between control and need for interaction would be to individually pick one song after the other;
• creating a playlist and playing it would be similar, but the required interaction is concentrated at the beginning, when the list is created;
• selecting an album would be similar but does away with the need to create the list in the first place;
• etc…

More variables could be taken into account. Of great importance is the perceived, subjective “quality” of the musical stream, how much it conforms to expectations, fits the user’s mood and follows an overall theme. Another one is the discovery of new music, which very naturally leads to a social component when we want to share our discoveries. The social facet of music does not stop at sharing: music is often used as a means of self-expression, as one of the ways that we use to create and present an image of ourselves to others. These concepts and more may determine the choices and trade-offs made during design of an interactive solution.

The ideas outlined here so far are based on some reading, personal experience and a few assumptions. Assuming is not necessarily bad but one needs to keep in mind that an assumption is not the same as hard knowledge: eventually, assumptions need to be challenged and tested. I will talk about this testing in future updates.