UIScape

As a way of enriching the way we interact with and perceive the physical spaces we live and work in, more and more information technology is being integrated in architecture. Video screens in elevators, bars that react to touch and buildings that let passers-by catch a glimpse of the activity inside are all examples of ambient displays. They provide peripheral information, are smoothly integrated into the physical environment and usually have a focus on aesthetic appeal.

A limitation of most ambient displays, and in fact of public displays in general, is that they are not personal: everyone gets to see the same information. This limits their possible applications, leaving hand-held devices as the only means of getting more personalised information.

However, there is a way around this. If a display “broadcasts” to the public by cycling through all the information people might need, individuals can tune in to the part they’re interested in by paying attention to the appropriate time slot in each cycle. Many public displays already do this kind of multiplexing. For example, train times may be shown on two alternating “pages” on a screen, or that display in the elevator may cycle through the weather forecast, news headlines and celebrity gossip. The problem here is that you need to watch constantly to pick out the parts you want. The interaction is no longer peripheral, instead becoming the main focus of your attention.

Insights from cognitive neuroscience into how our brain can integrate information from two different senses, or modalities, come to the rescue here. Researchers from the University of Newcastle upon Tyne realised that you could cue users through a modality other than vision to guide their attention towards the right time slot in a display’s cycle.

They designed a navigation system called CrossFlow, which projects arrows onto the floor, pointing in each of the possible directions in turn, in a repeating cycle. To know which set of arrows to follow, a user specifies their destination on a mobile device. The device then figures out the schedule of the relevant arrows, and vibrates and/or beeps in sync with them. This cross-modal cue allows the user to focus on a particular direction, without having to pay constant attention to either the ambient display or the mobile device.

Testing of the system against using a map showed improved performance both in navigating and in arithmetic tasks they had to do at the same time, and participants perceived their mental workload to be lower.

I find something strangely elegant and compelling about this concept of a public-private information display. To bystanders, the public, visual component of the display presents a mysterious and aesthetic phenomenon. Only those who receive the other half of the information in the form of haptic or auditory cues can make sense of it. And, as long as only vibration is used, you won’t know how the person standing next to you is perceiving it all.

More and more computing power is being put into handheld devices like mobile phones. A wealth of applications that were previously only available on desktop computers are now available in the palm of your hand. Taking these applications with you on the move is often as useful as the designers hope, but once as I tried to complete a game of Monopoly in London’s Victoria Station at rush hour, I knew that I was doing two things that were fundamentally incompatible with each other. 

A group of Finnish researchers have asked the question: just how much does a busy, demanding real-world context affect the user’s interaction with mobile devices? The answer, in case you’re walking across Victoria Station too, is a lot.

In fact, this study clearly demonstrates that mobility itself - the very thing we value mobile phones for - directly limits our ability to use mobile applications.

The researchers argue that we can‘t be mobile without dedicating some of our precious cognitive resources to maintain our position in the world. We have to commit social tasks like maintaining a sense of privacy on a crowded train, by monitoring the position of others and then shifting position appropriately. We also have to commit navigational tasks when mobile, like finding our current location, planning a route, walking purposely to avoid oncomers, buses, and so forth. Simply put, the time we spend thinking about these things means we can’t think about the device we’re trying to use at the same time.

                                                   ^{Image courtesy of evanrude}

This paper gives an idea of just how much our attention falls away in demanding mobile settings. In the lab, the participants looked at the mobile screen for an average of about 14s before looking away. When the participants were on a busy street, it was only 4s - less than a third of the time. This pattern continued throughout many measures: the strongest in my opinion is that in the lab, participants looked at their external environment for only 5% of the time they were using the device, while in the busy street participants were distracted enough that they looked away from the screen 51% of the time.

Two small quibbles with these figures is that they only measured visual attention, and only while a new internet page was loading. We can attend to our external environment in ways other than looking at it, and we’re more likely to look away from the screen while it’s loading than at other times. So I’d say keep in mind the ratios between the figures, rather than the absolute values.

And clearly, the users were in a difficult position: trying to use this new device while not walking into anyone or anything. How did they cope with these conflicting demands? The researchers reported that users seemed to pay lots of attention to a setting just after they entered it, then settled into long periods of using the device, punctuated by brief periods of attending to the environment. Perhaps most interesting, the researchers noted that when the environment demanded social interaction, users almost always stopped using the mobile device until the social demand was fulfilled. This hints that users give social demands a much higher priority than using their device.

For those of us looking to design, this paper is worth knowing about because it provides some hard guidelines to a previously intuitive idea of mobile contexts. For instance, don’t dream that your users spend as long as you do looking at the screen: count on 4s of your user’s time before they have to look away. You might get half your users time if they’re busy, so respect what time you get. Provide a simple, fast, easy-to-scan interface. Let them finish quickly, so they can get on with the more pressing task of making their way across Victoria station.

It is often said that a picture is worth a thousand words. In the case of a picture created through photography, what would these thousand words actually describe? A photograph is most often a static depiction of a scene at one specific moment in time. Although with modern cameras it is relatively easy to capture all the visual detail of a scene, this is rarely sufficient for portraying the context in which this scene was set, which is usually left to the imagination of the viewer.

A group of researchers from the Future Applications Lab at the Viktoria Institute in Sweden set themselves to changing this status quo by proposing context photography, a novel way of representing within the photo itself the context in which it was taken. In their experimentations, they chose two parameters to represent context: camera movement and ambient sound. Their prototype “context camera”, implemented using a camera-phone, monitors these parameters in real time and feeds them into a set of visual effects that are applied to the picture as is it shot with the phone camera. This process is adjustable by the photographer so that, for example, one can chose to associate the effect of colour shadows with the presence of high-pitched ambient noise. Some examples of the resulting photographs are visible in the following pictures:

In their latest research paper published in the 2006 NordiCHI conference, the researchers describe the reactions of photographers who used context camera-phone prototypes for a period of six weeks and submitted a total of around 300 pictures. Although each participant’s experience with context phtography was unique, possibly reflecting their different attitudes to personal photography in general, some common themes did emerge from the participants’ photographs and comments

First, as with many innovative interactive technologies, context photography has found unexpected uses. Although it was probably conceived as a way of capturing the existing context, it was found that users would attempt to artificially “create” some context in order to trigger the contextual effects. For example, some users would scream if there were no ambient sounds, or would try moving the camera in different ways if there was no natural movement in the scene. In doing so, they turned the contextual parameters into yet another input in the creating process of photography.

However, users would not generally agree how these inputs should affect the resulting image. The aesthetics of the visual effects applied by the context camera proved to be “highly subjective and […] a matter of personal taste”. Although each photographer could adjust the visual effects through some calibration process, some found this process complicated or ambiguous. The researchers themselves concede that “designing effects to suit a high number and wide range of users becomes a challenging task”.

Although it may take some time for context photography to be optimised for the everyday user, Apple’s iPhone is already pioneering the use of sensors that measure ambient light, body proximity and acceleration, and could become an interesting platform for context photography experimentation. How about a mobile, context photography-based version of Apple Photo Booth effects?