Screen-based user interfaces now include dynamic and moving elements that transform the screen space and relations of mediated content. These changes place new demands on design as well as on our reading and use of such multimodal texts. Assuming a socio-cultural perspective on design, we discuss in this article the use of animation and visual motion in interface navigation as navimation. After presenting our Communication Design framework, we refer to relevant literature on navigation and motion. Three core concepts are introduced for the purpose of analysing selected interface examples using multimodal textual analysis informed by social semiotics. The analysis draws on concepts from multiple fields, including animation studies, ‘new’ media, interaction design, and human-computer interaction. Relations between time, space and motion are discussed and linked to wider debates concerning interface design.

Let’s look at some of the videos.

Synne Frydenberg

This idea shows quite directly how the properties of paper can be used in motion sketching, in this case by folding the paper.

Natasha Ruivo:

Natasha makes use of what I call virtual kinetics, by giving the sensation of gravity and mass as the images fall down.

This video shows nicely how stop motion can be used to show how a specific task or action in the interface can be accomplished. This would be much harder to sketch and present without such a video.

Fanny Monier

Fanny shows how stop motion can be used to create a sensation of transformation and three-dimensional space by simply substituting one drawing with another. This technique seems highly effective.

Another way to create the sensation of a three-dimensional space is of course to physically set it up in three dimensions. This efficiently takes us away from the page metaphor and allow us to rather conceptualise the interface as a stage or a theatre.

Svein Inge Bjørkhaug

The idea behind this video is to make a way of sorting images as they are imported from a digital camera to a computer. I think the motion sketch communicates quite well how this idea could be realised, by showing how the user actions are linked to what happens on the screen.

Ivan Milanovic

Ivan was the only student working with a real device, the iPhone. It must have been quite hard to work with such small elements and the small screen space. The first part of the video is a bit too rough, but the last part of the video shows nicely how a tilting gesture can be used to move between images.

Theo Tveterås

A simple but nice idea: touch the surface, and images appear underneath each finger.

As Theo argued, the principle that is sketched out here could be used in a screen interface by using other ways of controlling the navigation. I also think it could be a fun way of navigating a device with screens on both sides!

A playful way of engaging with photos: a memory game!

What did we learn?

Stop motion prototyping obviously has its advantages and disadvantages. These are the main advantages:

• quite easy and quick to prototype visual elements that move, especially in three dimensions.
• easy to show how user action and screen presentation may be related over time
• the rough style of stop motion is suited for rough sketching
• physical space and physical elements (such as paper) may inspire and induce new ideas. Great potential for experimentation.
• effective for communicating and presenting ideas

The challenges:

• requires some equipment and software tools (camera, tripod, light, PC, software)
• it may take some time to prepare the visual elements (i.g. paper pieces)
• there are limits for what you can easily do in front of a camera. Some things are more easy to do on a computer
• it is hard and time consuming to produce subtle and sophisticated movement by using stop motion

Social Navimation is the phenomenon of engaging with social media through a navimational interface, or more specifically, to intertwine navigation with motion in
social media applications. In this paper I analyse a set of design examples of how social navimation can be realised.

I got a lot of positive comments after the presentation, especially on the visual presentation. Overall I am happy with the Prezi tool, even if I have discovered some bugs. It took me quite some time to make the presentation, but now I know the tool and will definitely use it for future presentations.

“Since they need to be able to capture the essence of design concepts around transitions, dynamics, fell, phrasing, and all the other unique attributes of interactive systems, sketches of interaction must necessarily be distinct from (traditional) types of sketches…” Bill Buxton: Sketching User Experiences

This week I have been involved in teaching a group of interaction design students at the Design for interactive and social media course at AHO. The topic has been ‘Sketching with time’, and has focused on using stop motion combined with paper prototyping to sketch interface ideas. The week’s assignment was to make a photo album interface and experiment with navimation.

Before introducing the students to the technique I had to try it out myself. I found the Mac application FrameByFrame which has been brilliant for this purpose. The functionality of the software is limited, but it is free, extremely simple to use, and serves the purpose for quick motion sketching.

Here are two of the quick stop motion sketches I made:

I also tried using video, recording my actions in real time:

The video quality is quite rough (partly because I am using a really old DV camera), but I don’t see this as a big problem. The technique is primarily to be used for quick sketches early in the design process.

The students got three days to make their video sketches. During these days many of the students managed to do a lot of experimentation and test out different ideas. The task was in many ways an experiment from our side, so I was positively surprised by the diversity and quality of their work. I also got the impression that they had learned a lot about timing, response and communication in the interface.

The technique has clearly some disadvantages – it is for example hard to make subtle movements and deal with details and many elements at the same time. However, it seems especially suited for 3D motion sketching, since this often requires a lot of time and skills to do on a computer.

UPDATE: see some of the videos the students made.

Are you compelled by the rapid development of mobile devices and their graphically sophisticated screens?

Have you noticed the increasing employment of visual motion in screen-based interfaces?

Are you interested in the possibilities and challenges for designing such interfaces?

Jørn Knutsen and I will arrange a full day workshop at the NORDES conference Engaging Artifacts in Oslo, August 30 – September 1, 2009. We will introduce participants to theoretical concepts and design techniques for prototyping screen-based interfaces that make use of visual movement. Participants will work hands-on in groups, exploring techniques for developing simple prototypes. The workshop ends with a general discussion in which we address theoretical as well as practical issues.

Full workshop description (PDF).

Conference program. NB: early registration until August 1.

UPDATE: The workshop was cancelled.

I will present a full peer-reviewed paper on the Nordic Design Research conference NORDES’09: Engaging Artefacts, which will take place in Oslo in the end of August this year. The paper is called ‘Social Navimation: Engaging Interfaces in Social Media’, and explores how visually dynamic interfaces can enhance social media applications. This potential is investigated through experimental design production, followed by a textual analysis of the resulting interface prototypes. The term ’social navimation’ is introduced and applied in the analysis, in which I investigate how semiotic resources from navimation are connected to features of social media. Hopefully, the paper will be of interest for both theory and practice of interface and interaction design, and new media studies in general.

UPDATE: The conference program and all papers are now available here. Download the full paper: Social Navimation: Engaging Interfaces in Social Media (PDF).

UPDATE 2: See the presentation.

The example is from a master project in industrial design at Konstfack by Karin Ehrnberger, called ‘Materializing gender’. I saw this presented on the design conference NORDES in Stockholm in 2007, and was impressed by how the theoretical discussion was interwoven with a specific design project. The paper is unfortunately not available online, but a short description can be found here.

The aim of the paper was to investigate how products are ‘gender coded’, and bring attention to the lack of understanding of this phenomenon. This was partly done theoretically by drawing on gender studies, but also through experimental design production. Ehrnberger designed a hand mixer using the aesthetics of a drill, and a drill using the aesthetics of a hand mixer (images used with permission from the designer).

Even though these artifacts do not convey a unambiguous argument in themselves, they are to my mind highly successful in visualising and bringing attention to how gender is semiotically “encoded” in physical products. These artefacts do not reduce the need for theory and analysis, rather they demonstrate the importance for a thorough and theoretical inquiry. In addition, these products have the ability to evoke reactions from both researchers, designers, and users, and thereby increase the potential for knowledge dispersion.

In my research I intend to do something similar, that is to experiment with form of academic communication. Rather than keeping the design production and the academic text separate, I want to integrate them in a coherent whole. To put it simple: I will discuss and analyse navimation through a navimational interface. I have already done a small experiment and plan to take this further in some way or another. This is not easy to do in regards to both design and research, but I think there is a great potential for innovation in academic communication here.

OnLive is also interesting in terms of interface and visual design. Just take a look at this video of the game interface:

As you can see, the service provides a visually rich interface. In the cinematic intro sequence, we fly through the logo and enter a new universe. The ‘virtual camera’ then flies over a globe that is filled with small tiles of videos, before it settles on the main menu. In navigating between different sections and games, a range of different animation techniques and transitions are employed. I looks like the rich environment will make it both easy and fun to find new games, friends, and other players.

I’m exited that the people at OnLive have developed a technological platform that reduces the disadvantages of employing advanced motion graphics, and that they have chosen to design such a rich navimational interface. I just hope it won’t be too long before we get this to Norway…

Even if Fallman takes on an impossible task for such a short paper, he offers very interesting concepts and arguments. The paper is written from the perspective of HCI, not design as such. However, he argues that HCI has emerged as a design-oriented field. Many of the arguments and concepts presented should be relevant for design research in general.

3 accounts on what design ‘is’

Fallman presents three accounts from design theory on what design ‘is’. He calls these the conservative, the romantic, and the pragmatic.

1. The conservative account sees design as a rational process that aims to convert an undesired situation into a desirable one, by going through rational and structured methodological steps. This account assumes that there is a ‘problem’ to be solved. Design is thought of as a scientific or engineering endeavour, and the focus is on normative design methods and generic design principles. For this account, references are given to H. Simon, C. Alexander, J. Löwgren and E. Stolterman.
2. The romantic account gives prominence to the role of the designer, which is seen as a mastermind or creative genius. Here, imagination and creativity is seen as key abilities rather than abstract reasoning and rational problem solving. Rather then a focus on the process, the focus is on values, taste, quality and aesthetics. Art is the role model here, rather then science. References are given to R. Coyne, E. Stolterman, and P. Louridas
3. The pragmatic account holds that design always is carried out in a specific situation, where the designers iteratively interpret the effects of their designs on the situation on hand. Design can therefore be seen as a hermeneutic process. This view draws on pragmatism and may see knowing-in-action as a specific and important kind of knowledge. Since every design situation is different, the ability to deal with different situations is more important than theories and methodology for guidance. Reference is given to D. Schön and D. Ihde.

I do not currently have the necessary overview to evaluate these categories, though I wonder if they may be a bit simplified and exaggerated. I do not think many designers (my self included) would position themselves in either one of these categories. In practice, it seems to me that design may sometimes include rational problem solving, sometimes ‘mysterious’ creativity, and most often the specificity of the situation at hand. The emphasis will vary from field to field, from designer to designer, and from project to project. However, such categories provide us with means for discussing what design ‘is’, which is essential in a theoretical inquiry. This understanding may further influence the way design education is conducted, and how the role of design and designers is understood at a general level.

Sketching as design thinking

To get closer to what design ‘is’, Fallman considers the role of sketching as an archetypical design activity. He argues that designers’ thinking is mediated by sketches and prototypes. The materials ‘talk back’ to the designer in an ongoing dialogue between the designer and the sketch.

…design involves the designer in a necessary dialogue with the materials of the design situation, from which the design problem and its solution are worked out simultaneously, as a closely coupled pair. (page 231)

In this unfolding process problem setting and problem solving (Schön) are seen as intertwined activities, in a search for a coherent, well-balanced whole.

[Design] … must not simply be seen either as a question of problem-solving, as an art-form, or as a bustle with reality: it is on the contrary an unfolding activity which demands deep involvement from the designer. ” (page 231-232)

Rather than seeing design as a hybrid discipline between art and science, design should be considered as a tradition guiding action and thought, which spans across many disciplines.

Design-oriented Research vs. Research-oriented Design

Fallman proposes a new way to distinguish research activities in HCI that involve design production.

• Design-oriented Research has knowledge of some sort as the main contribution. This is the conduct of academic researchers, and should be conducted when the knowledge would not be attainable if design production was not a vital part of the research process. Here, the designed artefact is considered a means to an end, for example for exploring possibilities outside a current paradigm.
• Research-oriented Design has production of new artefacts as its main motivation, not the production of new knowledge. Nevertheless, this production may relate to research in many ways. I suppose what is normally considered development would fit within this category.

Fallman argues that Design-oriented Research should include problem setting as an important part, while Research-oriented Design most often has problem solving as its main component.

Relating this to my own work, it is clear that doing a PhD entails scholarly and academic research where the aim is to produce new theoretical knowledge. The production of artifacts is subordinate. However, I think the production of artefacts can play an important role not only in informing theory, but also in embedding, presenting and visualising theory. In a social-cultural view on artefacts, the artefact does not in itself have a determined meaning. Nontheless, an artefact may play an important role in the dispersion of knowledge, as long as the theoretical argument is made clear and apparent.