UIScape

Have you ever stood in front of one of those dual-30″ Cinema Display setups in an Apple Store and wondered whether you’d get a stiff neck working with so much screen real-estate? With desktop display sizes growing quickly, and more and more employers recognising that dual-screen setups can increase productivity, it’s actually becoming a valid question whether there’s a limit past which this trend becomes unreasonable. In certain scientific and military applications, visualisations are already big enough to require physically walking from one part of the display to another.

At last week’s CHI ’07 conference, two studies from Virgina Tech were presented that fit into this theme. The first one looked into what would happen to users’ performance if a display was so big that it required walking. They tested both a spatial, map-based visualisation and a more abstract grid-based design, at 2560 × 768, 5120 × 1536 and a whopping 10240 × 3072 pixels (about 2.7m × 1.0m or 9′ × 3.5′). The tasks on the larger displays involved more data, and so would be expected to take longer. But it seems that our ability to process visual information scales quite well: people took on average only about three times as long when the visualisation was sixteen times larger (with variation between tasks).

The second study also tested different display sizes (the largest one being the same as in the other study), but with the aim of comparing physical navigation to its “virtual” counterparts, panning and zooming. With the larger displays, participants tended to rely less on virtual navigation, showing that people do in fact prefer moving around or turning their head. This turned out to be the right choice, as it was also more efficient than panning and zooming.

Putting these results together, it would seem that having a larger display always pays off in terms of cognitive efficiency, navigation efficiency and user preference, even if it’s too big to see all at once. Interestingly, both studies found that spatial visualisations benefit more from the extra screen real-estate than non-spatial ones.

Although efficiency is important, it would also be interesting to see a physical ergonomist’s take on the issue. Do extra-large displays hold new risks of work-related injuries, or is the extra movement actually healthier than our traditionally static workstations?

Almost all science fiction movies made so far have invariably featured some form of speech recognition, where all sorts of computers, robots or even sliding doors could accurately understand and respond to human speech under any conditions. Although speech recognition technology hasn’t progressed quite as fast as SF writers had imagined, it is increasingly being used for everyday products and services: a handful of mobile phones can already respond to basic voice commands without needing any previous training with samples of your voice, and Interactive Voice Response (IVR) systems will pick up your call, listen to what you want, and tell you where is the nearest restaurant or when is the next train.

The reason such applications have been (relatively) succesful is that they manage to overcome the inaccuracies of speech recognition and the ambiguity of human speech by limiting themselves to a particular domain (e.g. requests about movie showtimes) and taking into account the specific domain knowledge (e.g. users are likely to pronounce the name of a movie currently on show). By carefully following this principle, researchers from the National Research Council of Canada have created VoiceCode, an application that allows computer programmers to dictate program code to their computer, with virtually no need to touch the keyboard.

The main issue with dictating program code is that programming languages were never meant to be spoken but were designed to have a clear, simple structure that can be easily understood by a computer. A simple example of this is the naming of functions and variables: in program code it is not uncommon to see abbreviations such nInMsgs for a variable that represents the number of incoming messages. Without VoiceCode, one would have to spell out the abbreviation somehow like “n-capital-i-n-capital-m-s-g-s”. VoiceCode, however, can learn and adapt to the ways that computer programmers abbreviate such names, and match a full spoken phrase with its likely abbreviations. But such heuristics are bound to fail once in a while, so what happens in this case? VoiceCode again follows a basic principle of interaction design: in case of error, give the user a quick way to recover. Whether the error was made in the stage of voice recognition or in translating natural language to program code, it can be easily corrected by selecting one of the alternative interpretations presented in a popup window.

It’s impossible to describe all the features that make this piece of software a great case study in voice recognition, so I highly recommend watching their demo video that was presented in the CHI 2006 conference. However, as impressive as this video may be, don’t rush to throw away your keyboards just now. You may escape RSI, but VoiceCode authors are quick to point out that a similar condition, voice strain may be linked to the continuous use of speech recognition products. One can only wonder if such problems may also arise with more futuristic input technologies, such as brain wave reading.

When working with digital content on a screen, we spend an awful lot of our time scrolling. Two things in particular can make this very ineffective. One is that you often want to traverse content linearly that is represented in two dimensions, for example some text that’s in several columns on a page. If your screen isn’t big enough to fit all columns, you end up having to scroll up and down and left to right repeatedly to read it.

The other problem is that a lot of the stuff you scroll through may not actually be important. If you’re interested in particular parts of a document, everything in between feels like a waste of space while scrolling.

Edward Ishak and Steven Feiner of Columbia University have devised a technique for dealing with these issues. Their solution is to identify the content of a document that’s relevant to a task and to determine a meaningful path through it, which the user can then move along with a special scrollbar. This achieves two things: the user’s one-dimensional action can be translated into movement through two-dimensional content, and “unimportant” areas can be skipped automatically. Actually, rather than simply skipping them, their system “flies” over these areas at high speed, while at the same time zooming out to help you keep your orientation.

You can see the the technique in action in this movie, which gives you a good idea of how it works. As part of their research, they implemented this for reading multi-column PDF documents, for jumping between search results in a text, and even for traversing all the faces in a photograph.

The content-aware scrolling path through a two-column text document, for search results and for faces in a photo. Dashed segments are flown over automatically.

The issues this design addresses are particularly pertinent to hand-held devices with small screens. Other approaches in this area include tilt-based scrolling, momentum-based scrolling and zooming. But even though scrolling in two dimensions may not be so common on today’s large desktop screens, finding a place in a long document is, and content-aware scrolling has the potential to help even here. Unfortunately, this first study didn’t include formal user testing, so the real-world usability of the technique is still uncertain.