UIScape

Have you ever stood in front of one of those dual-30″ Cinema Display setups in an Apple Store and wondered whether you’d get a stiff neck working with so much screen real-estate? With desktop display sizes growing quickly, and more and more employers recognising that dual-screen setups can increase productivity, it’s actually becoming a valid question whether there’s a limit past which this trend becomes unreasonable. In certain scientific and military applications, visualisations are already big enough to require physically walking from one part of the display to another.

At last week’s CHI ’07 conference, two studies from Virgina Tech were presented that fit into this theme. The first one looked into what would happen to users’ performance if a display was so big that it required walking. They tested both a spatial, map-based visualisation and a more abstract grid-based design, at 2560 × 768, 5120 × 1536 and a whopping 10240 × 3072 pixels (about 2.7m × 1.0m or 9′ × 3.5′). The tasks on the larger displays involved more data, and so would be expected to take longer. But it seems that our ability to process visual information scales quite well: people took on average only about three times as long when the visualisation was sixteen times larger (with variation between tasks).

The second study also tested different display sizes (the largest one being the same as in the other study), but with the aim of comparing physical navigation to its “virtual” counterparts, panning and zooming. With the larger displays, participants tended to rely less on virtual navigation, showing that people do in fact prefer moving around or turning their head. This turned out to be the right choice, as it was also more efficient than panning and zooming.

Putting these results together, it would seem that having a larger display always pays off in terms of cognitive efficiency, navigation efficiency and user preference, even if it’s too big to see all at once. Interestingly, both studies found that spatial visualisations benefit more from the extra screen real-estate than non-spatial ones.

Although efficiency is important, it would also be interesting to see a physical ergonomist’s take on the issue. Do extra-large displays hold new risks of work-related injuries, or is the extra movement actually healthier than our traditionally static workstations?

As a way of enriching the way we interact with and perceive the physical spaces we live and work in, more and more information technology is being integrated in architecture. Video screens in elevators, bars that react to touch and buildings that let passers-by catch a glimpse of the activity inside are all examples of ambient displays. They provide peripheral information, are smoothly integrated into the physical environment and usually have a focus on aesthetic appeal.

A limitation of most ambient displays, and in fact of public displays in general, is that they are not personal: everyone gets to see the same information. This limits their possible applications, leaving hand-held devices as the only means of getting more personalised information.

However, there is a way around this. If a display “broadcasts” to the public by cycling through all the information people might need, individuals can tune in to the part they’re interested in by paying attention to the appropriate time slot in each cycle. Many public displays already do this kind of multiplexing. For example, train times may be shown on two alternating “pages” on a screen, or that display in the elevator may cycle through the weather forecast, news headlines and celebrity gossip. The problem here is that you need to watch constantly to pick out the parts you want. The interaction is no longer peripheral, instead becoming the main focus of your attention.

Insights from cognitive neuroscience into how our brain can integrate information from two different senses, or modalities, come to the rescue here. Researchers from the University of Newcastle upon Tyne realised that you could cue users through a modality other than vision to guide their attention towards the right time slot in a display’s cycle.

They designed a navigation system called CrossFlow, which projects arrows onto the floor, pointing in each of the possible directions in turn, in a repeating cycle. To know which set of arrows to follow, a user specifies their destination on a mobile device. The device then figures out the schedule of the relevant arrows, and vibrates and/or beeps in sync with them. This cross-modal cue allows the user to focus on a particular direction, without having to pay constant attention to either the ambient display or the mobile device.

Testing of the system against using a map showed improved performance both in navigating and in arithmetic tasks they had to do at the same time, and participants perceived their mental workload to be lower.

I find something strangely elegant and compelling about this concept of a public-private information display. To bystanders, the public, visual component of the display presents a mysterious and aesthetic phenomenon. Only those who receive the other half of the information in the form of haptic or auditory cues can make sense of it. And, as long as only vibration is used, you won’t know how the person standing next to you is perceiving it all.

More and more computing power is being put into handheld devices like mobile phones. A wealth of applications that were previously only available on desktop computers are now available in the palm of your hand. Taking these applications with you on the move is often as useful as the designers hope, but once as I tried to complete a game of Monopoly in London’s Victoria Station at rush hour, I knew that I was doing two things that were fundamentally incompatible with each other. 

A group of Finnish researchers have asked the question: just how much does a busy, demanding real-world context affect the user’s interaction with mobile devices? The answer, in case you’re walking across Victoria Station too, is a lot.

In fact, this study clearly demonstrates that mobility itself - the very thing we value mobile phones for - directly limits our ability to use mobile applications.

The researchers argue that we can‘t be mobile without dedicating some of our precious cognitive resources to maintain our position in the world. We have to commit social tasks like maintaining a sense of privacy on a crowded train, by monitoring the position of others and then shifting position appropriately. We also have to commit navigational tasks when mobile, like finding our current location, planning a route, walking purposely to avoid oncomers, buses, and so forth. Simply put, the time we spend thinking about these things means we can’t think about the device we’re trying to use at the same time.

                                                   ^{Image courtesy of evanrude}

This paper gives an idea of just how much our attention falls away in demanding mobile settings. In the lab, the participants looked at the mobile screen for an average of about 14s before looking away. When the participants were on a busy street, it was only 4s - less than a third of the time. This pattern continued throughout many measures: the strongest in my opinion is that in the lab, participants looked at their external environment for only 5% of the time they were using the device, while in the busy street participants were distracted enough that they looked away from the screen 51% of the time.

Two small quibbles with these figures is that they only measured visual attention, and only while a new internet page was loading. We can attend to our external environment in ways other than looking at it, and we’re more likely to look away from the screen while it’s loading than at other times. So I’d say keep in mind the ratios between the figures, rather than the absolute values.

And clearly, the users were in a difficult position: trying to use this new device while not walking into anyone or anything. How did they cope with these conflicting demands? The researchers reported that users seemed to pay lots of attention to a setting just after they entered it, then settled into long periods of using the device, punctuated by brief periods of attending to the environment. Perhaps most interesting, the researchers noted that when the environment demanded social interaction, users almost always stopped using the mobile device until the social demand was fulfilled. This hints that users give social demands a much higher priority than using their device.

For those of us looking to design, this paper is worth knowing about because it provides some hard guidelines to a previously intuitive idea of mobile contexts. For instance, don’t dream that your users spend as long as you do looking at the screen: count on 4s of your user’s time before they have to look away. You might get half your users time if they’re busy, so respect what time you get. Provide a simple, fast, easy-to-scan interface. Let them finish quickly, so they can get on with the more pressing task of making their way across Victoria station.