by Amar
Content-aware scrolling
When working with digital content on a screen, we spend an awful lot of our time scrolling. Two things in particular can make this very ineffective. One is that you often want to traverse content linearly that is represented in two dimensions, for example some text that’s in several columns on a page. If your screen isn’t big enough to fit all columns, you end up having to scroll up and down and left to right repeatedly to read it.
The other problem is that a lot of the stuff you scroll through may not actually be important. If you’re interested in particular parts of a document, everything in between feels like a waste of space while scrolling.
Edward Ishak and Steven Feiner of Columbia University have devised a technique for dealing with these issues. Their solution is to identify the content of a document that’s relevant to a task and to determine a meaningful path through it, which the user can then move along with a special scrollbar. This achieves two things: the user’s one-dimensional action can be translated into movement through two-dimensional content, and “unimportant” areas can be skipped automatically. Actually, rather than simply skipping them, their system “flies” over these areas at high speed, while at the same time zooming out to help you keep your orientation.
You can see the the technique in action in this movie, which gives you a good idea of how it works. As part of their research, they implemented this for reading multi-column PDF documents, for jumping between search results in a text, and even for traversing all the faces in a photograph.
The content-aware scrolling path through a two-column text document, for search results and for faces in a photo. Dashed segments are flown over automatically.
The issues this design addresses are particularly pertinent to hand-held devices with small screens. Other approaches in this area include tilt-based scrolling, momentum-based scrolling and zooming. But even though scrolling in two dimensions may not be so common on today’s large desktop screens, finding a place in a long document is, and content-aware scrolling has the potential to help even here. Unfortunately, this first study didn’t include formal user testing, so the real-world usability of the technique is still uncertain.
- Categories: ergonomics, graphical interfaces, mobile technology
- Trackback from your site
17:34
I think I might find this most confusing, as I’d want to know what parts of the document I’m missing.
I wonder if other people might be too curious, like me, for this technique to be of use.
01:18
Interesting, but I find the multi-column thing a bit of a red herring. Text is *printed* in multiple columns to give a readable line width, and to fit multiple stories onto a page of fixed dimensions.
On a computer screen, pages can have infinite dimensions, and there’s little reason to include the entire text of multiple stories on one viewable area. A readable line-width is easily achievable.
Seems like it’d be easier to just use PDFs for print, and convert them to a nicer computer format (e.g. HTML, like this very blog) for reading on computer.
19:32
pauldwaite brings up a good point, however I still think there are cases when a document is formatted to be printed but is read on-screen. Even though it’s true that web pages have infinite dimensions, and ideally we’d like to have one format for each, it’s still much easier to just create one document that works printed, or on-screen, so I think this method is an interesting way of addressing that.
But I think an even better use of this technology is to create non-linear scrolling paths. For instance you could have a single piece of work and include multiple paths through the same document depending on what you’re interested in seeing; all while using a single scrollbar.
19:45
josh: having different paths through a document is also where I see a lot of potential.
For example, similar to how they demonstrated moving between search results, you could have modes for traversing all section headings, or all the images in a document.
23:34
Amar:
Or even, imagine telling multiple stories within one piece (I hesitate to use the word ‘document’ because ot conjurs ideas of text-heavy manuscripts which is misleading). It’s really about creating an experience which the author can control by setting the flow without having to reorganize the information on the page (which, perhaps is designed optimally for printing, or some other purpose).
The author (or even the user) has the power to define teh experience based on whatever parameters are important to them for the given context.
11:36
It’s an intresting thought, but it will take some time getting used to. It’s quite a fundamental change. It would be great to be able to get an overview of a document like this - flying over all the headings and importnt areas is a good idea.
It’d be interesting to see if they implement paths of different intensitiy. Say I want to read a text quickly, so I choose a quick path which show me all relevant sections for my understanding.
Then you can choose paths that go into mroe detail of the text - much like what the summarize sevice in Mac OS X does.
06:13
[…] UIScape. Tags: reading, software, ui, […]
17:19
This concept seems interesting at first glance, but I can see at least one definitely non-trivial issue, namely “to identify the content of a document that’s relevant to a task”. Oops! The definition of “task” in my world still lies in the user’s domain, not in the author’s (and even less in the machine’s). So, how is the relevance going to be determined (if not by a specific search)? The problem here – which is anything but technical – is that what somebody else *intends* to be relevant does not need to be the least relevant to me. And I tend to consider this kind cognitive preformation quite problematic – the value of serendipity is not to be underestimated. Nonetheless, I recognise that there might be enough cases where my argument does not prevail.
Something else: Why stick to the scrolling paradigm? Scrolling is fine if you want a *continuity* of context, i. e. linear access. But as far as I understand we’re dealing with random access in this case. Why not use e.g. a combination of several “layers” and fade over between them, perhaps combined with zooming? Think of a map of the document (in a broader sense) with simultaneous contextually triggered magnification – you could still scroll through the magnified areas of this map and zoom in your area of interest. Mind you, I do not claim that this was encessarily a better solution. But I would like to point out that it seems important to me to take a step back and consider other possibilities outside the given metaphor of representation. Never forget that it is not a physical object that is dealt with but *data*. And data does not have any “natural” appearance but is more or less infinitely malleable.