UIScape

Have you ever stood in front of one of those dual-30″ Cinema Display setups in an Apple Store and wondered whether you’d get a stiff neck working with so much screen real-estate? With desktop display sizes growing quickly, and more and more employers recognising that dual-screen setups can increase productivity, it’s actually becoming a valid question whether there’s a limit past which this trend becomes unreasonable. In certain scientific and military applications, visualisations are already big enough to require physically walking from one part of the display to another.

At last week’s CHI ’07 conference, two studies from Virgina Tech were presented that fit into this theme. The first one looked into what would happen to users’ performance if a display was so big that it required walking. They tested both a spatial, map-based visualisation and a more abstract grid-based design, at 2560 × 768, 5120 × 1536 and a whopping 10240 × 3072 pixels (about 2.7m × 1.0m or 9′ × 3.5′). The tasks on the larger displays involved more data, and so would be expected to take longer. But it seems that our ability to process visual information scales quite well: people took on average only about three times as long when the visualisation was sixteen times larger (with variation between tasks).

The second study also tested different display sizes (the largest one being the same as in the other study), but with the aim of comparing physical navigation to its “virtual” counterparts, panning and zooming. With the larger displays, participants tended to rely less on virtual navigation, showing that people do in fact prefer moving around or turning their head. This turned out to be the right choice, as it was also more efficient than panning and zooming.

Putting these results together, it would seem that having a larger display always pays off in terms of cognitive efficiency, navigation efficiency and user preference, even if it’s too big to see all at once. Interestingly, both studies found that spatial visualisations benefit more from the extra screen real-estate than non-spatial ones.

Although efficiency is important, it would also be interesting to see a physical ergonomist’s take on the issue. Do extra-large displays hold new risks of work-related injuries, or is the extra movement actually healthier than our traditionally static workstations?

As a way of enriching the way we interact with and perceive the physical spaces we live and work in, more and more information technology is being integrated in architecture. Video screens in elevators, bars that react to touch and buildings that let passers-by catch a glimpse of the activity inside are all examples of ambient displays. They provide peripheral information, are smoothly integrated into the physical environment and usually have a focus on aesthetic appeal.

A limitation of most ambient displays, and in fact of public displays in general, is that they are not personal: everyone gets to see the same information. This limits their possible applications, leaving hand-held devices as the only means of getting more personalised information.

However, there is a way around this. If a display “broadcasts” to the public by cycling through all the information people might need, individuals can tune in to the part they’re interested in by paying attention to the appropriate time slot in each cycle. Many public displays already do this kind of multiplexing. For example, train times may be shown on two alternating “pages” on a screen, or that display in the elevator may cycle through the weather forecast, news headlines and celebrity gossip. The problem here is that you need to watch constantly to pick out the parts you want. The interaction is no longer peripheral, instead becoming the main focus of your attention.

Insights from cognitive neuroscience into how our brain can integrate information from two different senses, or modalities, come to the rescue here. Researchers from the University of Newcastle upon Tyne realised that you could cue users through a modality other than vision to guide their attention towards the right time slot in a display’s cycle.

They designed a navigation system called CrossFlow, which projects arrows onto the floor, pointing in each of the possible directions in turn, in a repeating cycle. To know which set of arrows to follow, a user specifies their destination on a mobile device. The device then figures out the schedule of the relevant arrows, and vibrates and/or beeps in sync with them. This cross-modal cue allows the user to focus on a particular direction, without having to pay constant attention to either the ambient display or the mobile device.

Testing of the system against using a map showed improved performance both in navigating and in arithmetic tasks they had to do at the same time, and participants perceived their mental workload to be lower.

I find something strangely elegant and compelling about this concept of a public-private information display. To bystanders, the public, visual component of the display presents a mysterious and aesthetic phenomenon. Only those who receive the other half of the information in the form of haptic or auditory cues can make sense of it. And, as long as only vibration is used, you won’t know how the person standing next to you is perceiving it all.

It is often said that a picture is worth a thousand words. In the case of a picture created through photography, what would these thousand words actually describe? A photograph is most often a static depiction of a scene at one specific moment in time. Although with modern cameras it is relatively easy to capture all the visual detail of a scene, this is rarely sufficient for portraying the context in which this scene was set, which is usually left to the imagination of the viewer.

A group of researchers from the Future Applications Lab at the Viktoria Institute in Sweden set themselves to changing this status quo by proposing context photography, a novel way of representing within the photo itself the context in which it was taken. In their experimentations, they chose two parameters to represent context: camera movement and ambient sound. Their prototype “context camera”, implemented using a camera-phone, monitors these parameters in real time and feeds them into a set of visual effects that are applied to the picture as is it shot with the phone camera. This process is adjustable by the photographer so that, for example, one can chose to associate the effect of colour shadows with the presence of high-pitched ambient noise. Some examples of the resulting photographs are visible in the following pictures:

In their latest research paper published in the 2006 NordiCHI conference, the researchers describe the reactions of photographers who used context camera-phone prototypes for a period of six weeks and submitted a total of around 300 pictures. Although each participant’s experience with context phtography was unique, possibly reflecting their different attitudes to personal photography in general, some common themes did emerge from the participants’ photographs and comments

First, as with many innovative interactive technologies, context photography has found unexpected uses. Although it was probably conceived as a way of capturing the existing context, it was found that users would attempt to artificially “create” some context in order to trigger the contextual effects. For example, some users would scream if there were no ambient sounds, or would try moving the camera in different ways if there was no natural movement in the scene. In doing so, they turned the contextual parameters into yet another input in the creating process of photography.

However, users would not generally agree how these inputs should affect the resulting image. The aesthetics of the visual effects applied by the context camera proved to be “highly subjective and […] a matter of personal taste”. Although each photographer could adjust the visual effects through some calibration process, some found this process complicated or ambiguous. The researchers themselves concede that “designing effects to suit a high number and wide range of users becomes a challenging task”.

Although it may take some time for context photography to be optimised for the everyday user, Apple’s iPhone is already pioneering the use of sensors that measure ambient light, body proximity and acceleration, and could become an interesting platform for context photography experimentation. How about a mobile, context photography-based version of Apple Photo Booth effects?