Objects are not Slaves. Envisioning an Aesthetic Approach to the Design of an Interactive Dialogue with Objects

Marco Spadafora, Ph.D. Candidate - marco.spadafora@polimi.itDesign Department

Annamaria Andrea Vitali, Ph.D. Candidate - annamariaandrea.vitali@polimi.itDesign Department

Margherita Pillan, Associate Professor - margherita.pillan@polimi.itDesign Department

In this paper we present a critical analysis of the state of the art in the field of interactive products and systems and we discuss the main paradigms driving the development of Internet of Things (IoT). By this analysis, we argue that the user centred design methodologies generally employed in the development of digital solutions are not sufficient in their own to favour the generation of meaningful creative concepts. Furthermore, we discuss some case studies and present some results we obtained by creating innovative concepts through a different design approach, focused on the aesthetic potentialities of digital technologies. The design of interactive objects and services is mainly aimed to develop solutions improving efficiency and automation in daily activities such as getting information, manage and schedule time, reduce energy consumption and so on. On the contrary, following Hassenzhal principles of the “aesthetics of friction” (2011), we made some experiments aimed to explore the levers of the aesthetics of interaction so to act on the processes of interaction and dialogue between users and objects. In the paper we report the results of these experiments and discuss the importance of extend the focus of designers beyond the constraints of usability.

Keywords

Aestehtics of interaction, Smart object, Experience design, Technology.

INTRODUCTION

The design of technology based products and systems is generally performed following user centered design approaches. User’s needs and usability requirements, expressed in terms of efficiency and optimization of daily activities, have always been the leading drivers of interaction design. The goal of producing a pleasurable experience with interactive solutions is generally approached in terms of simple access to functionalities while the aesthetic dimension of projects usually refers to the material appearance of physical devices. Today we cope with the widespread of innovative “smart” mobile technologies, with sensors and the so called “internet of things”; despite the creative potentialities of technical inventions, most solutions are still characterized by old and stereotyped interaction paradigms, and produce trivial functionalities with flavorless design effects, mainly aimed to the automation of the control of objects. Conversely, we support the idea that traditional interaction design approaches are not enough to fully explore the entire potentialities of “smart” technologies and in this paper we present a design approach that finds inspiration in one fundamental pillar of design tradition: aesthetics. In the design of tangible products, aesthetics is expressed in terms of form, materials, colors, surfaces. On the other hand, in the design of smart applications and systems, the aesthetic quality perceived in the fruition is significantly related to intangible formal attributes such as the mechanisms of the interactive processes, the styles of dialogue between users and objects, the cause and effects dynamics; as designers, we should focus more an “aesthetics of interaction” since it is the main factor that differentiate a solution from another. Meanwhile we strongly believe that a design approach focused on the aesthetic quality of interaction would improve the quality of smart objects and applications and would open the way to a more meaningful and gratifying interpretations of smart technologies, far beyond usability and simplicity requirements. Objects should not be considered as “slave tools” at our disposals: they can have memory, provide and share information, extend our senses, support us in the extraction of meanings. We are converging toward the design of artifacts dialoguing with people and exploring different paradigms of these conversations. The paper provides an overview of the state of the art, and it enlightens novel approaches and theories overcoming simply usability requirements. Furthermore, we describe some design experiments that we performed so to explore innovative projects methodologies in the design of interactive objects and systems. We aim to set up the basis for a novel conception of smart technological products and services, beyond automation and efficiency, and to give a contribution to the development of a new generation of interactive solutions focused on the aesthetics of action.

RELEVANT WORKS AND THEORETICAL FOUNDATION

AESTHETICS, DESIGN, TECHNOLOGY

Following tradition, in the design of a material objects such a chair, designers take into account functional requirements, ergonomics, sustainability and other factors, but the role of aesthetics is central if not prior. As stated by Folkmann (2010), following a phenomenological approach, formal attributes elicit sensuous experiences, contribute to enrich perception effects and the act of communication/understanding between the subject and the objects and between subjects and the environment/reality. Moreover, by acting on perception and forms, designers intensify the presence of artifacts and the sense of placing objects in a specific context. Using some designed chairs as an example, Folkmann (2010) underlines how designers very often do not focus on functional requirements, but they manifest an abstract idea through sensuous appeals, and add a “surplus meaning”. To support this idea, he defined a framework in which pure functionality is seen as opposite of aesthetics: this categorization is related to the presence or not of surplus meaning and of designer’s idea expressed through sensuous and appearance. Trivial objects offering pure functionalities with anonymous form and natural mapping have no surplus meaning and Folkmann (2010) suggests to consider as a challenge for designers the creation of a “sensual universe” through the design of meaningful objects.

Talking about digital technologies, we observe that design has been mainly focused on “good” functioning and on rapid understanding of commands so to make fluid the relationship between technologies and people while usability, adherence to mental models and standardize practices have been the main rules to follow.

Technological objects are usually interpreted as means to accomplish tasks and everyday activities through interfaces. Jonas Lowgren (2013) defined interaction design as the act of “shaping digital things for people’s use”.

In this paper we argue the emergence of a different approach, so to transfer traditional approaches in design of digital objects.

The design of Experience contributes to explain how we can design interactive products taking into account pleasure beyond functionality, but rarely this approach goes beyond pleasure intended as simplification with the exception of some emerging researches that we name in the next paragraph.­ As designers aimed to produce meanings, we point out that simplified experiences are only a little part of the aesthetic possibilities we can imagine.

Going back to tradition, Achille Castiglioni designing Sella, synthetizes complex meanings through formal attributes that afford a specific way to experience a chair. In an analog way, in the design of interactive artifacts, we should experiment different languages and senses. We should embrace limits of digital technologies exploring its aesthetics potential – aesthetics of interaction - as we do with materials and productions technologies in the design of a chair. We argue that the exploration of the intertwinements between design, aesthetics and technology is the correct approach to design meaningful interactive products, able to appeal senses and produce meanings.

AESTHETICS AND INTERACTIVE PRODUCTS: BEYOND USABILITY

Some researchers have explored with different perspective the design approaches focused on aesthetics of interaction the consequences of considering interactive processes as a design subject.

Sensors and other electronic components transform objects into interactive entities conveying information (Backlund et al., 2006). As an example, the “transformational products” (Laschke, et al., 2011) provide alternative ways to perform basic activities such as remember or observe contexts, or elicit experiences through the use of a product (Lenz et al., 2013). These experiments go beyond the idea of using technology to solve problems such as seeing for unpaired people, and offer new design paradigms. In the analysis of these cases, we identified two main conceptions providing the theoretical foundation for our approach that is focused on smart technologies. Both focus on experience instead of ­functions.

Technological solutions supporting reflection moments

The first example we present is “slow technology” as defined by Hallnas et al. (2001).The authors advocate for the emergence of slow technology “to promote moments of reflection and mental rest in a more and more rapidly changing environment”. In their opinion, instead of developing technological ways to optimize time consumption related to a task, we can design for slow technology, i.e. we can design interactive artifacts that underline the presence of time, creating moment for reflection. Embracing the expressive features of digital technologies and disrupting stereotypes, we can create technology-based solutions that provide “time to reflect and think”. Their work Static! (Backlund et al., 2006), explores the aesthetic of energy and use displays to envision energy consumption (for example through luminescent cables) and to stimulate reflection in users.

From this perspective, interactive products and technologies can be seen as a mean to produce sense effects.

Hassezhal (2011) defined these new paradigms as a change in the aesthetic perspective, and as a passage from the “aesthetics of convenience”, (the traditional design based on usability requirements), to an “aesthetics of friction” toward transformational products.

We can therefore design products that create friction in the interaction process, that offer non trivial choices and require non obvious decision processes. Hassenzahl et al. (2014) defined designers as “troublemakers” instead of “trouble solvers”: technologies can be employed to enlighten critical decision moments, and to create opportunities of re-considering the sense of daily actions. In “Forget me not” (Laschke, et al., 2011), as an example, a desk lamp is designed so to turn-on only when people touch it, and it is programmed to slowly turn off after a lapse of time, as if it was saying: “hey, if you want more light, you have to ask, and be aware: asking for light, you are consuming energy”. Aesthetics of friction is based on the capability to give personality to interactive processes and virtual gestures, so to perturb usual experiences and adding meanings to it.

Both paradigms, slow technology and those inspired to the aesthetics of friction, require a new approach to interaction and experience design. Indeed, they provide the opportunity of designing products “with an attitude”, with an idea with a sensuous load that create a difference between them and any other one doing the same function.

INTERACTIVE ATTRIBUTES AND EMERGING EXPERIENCES

Some other studies focus their attention on the way we interact with products and the emerging experience. Lim et al. (2009) and Lenz et al. (2013) as an example, underline how different interactive attributes can actually affect emotions and perception. Lim focuses on digital interactive prototypes, and underlines the importance of the single elements of interaction– such as speed, frequency, alternatives - in affecting the perceived “mood” of the artifact.

Lenz et al (2013), identify three levels in the experience design, identified with “What”, “How” and “Why”. They stress the importance of working on the How of interaction: “high-level experiences are always mediated and created through the arrangement of concrete interactions on a lower level, i.e., the How of interaction. Creating meaningful experiences on the Why-level, thus, requires (1) an awareness of potential attributes of interaction on the How-level, and (2) reflection about which attributes may lead to which particular experience. […] Our approach, thus, explicitly differentiates the How (concrete interaction on a sensomotoric level) from the Why (experiences and meaning emerging from or supported through the interaction).”

TOWARDS A DESIGN APPROACH FOCUSED ON THE AESTHETICS OF INTERACTION

The brief overview here presented supports the idea that the design of meaningful interactive artefacts requires to re-define the way in which we seek balance and synthesis between function and form. Instead of working for usability and standardized ways of dialog between users and devices, we must play with technology so to understand the levers of expression and differentiation between two similar products. Differentiation is achievable exploring innovative How-s of interaction as a way to express sense. If we consider smart object and systems, we consider the question: what if smart solutions are not designed to be silently efficient, but to create engaging dialogues with people using all their expressive potentialities?

Let us consider an example of smart application, like the weather app Solar.

FIGURE 1a, 1b - Interfaces from Solar weather application (http://thisissolar.com/) FIGURE 1c - Interfaces of traditional weather application, like meteo.it

This app, similarly to any other weather forecast application, shows the weather forecast for the next days. However, instead of using traditional icons, symbols and filters, in Solar, the interface uses shades of a single colour to indicate the temperature (the warmer is the temperature, the warmer is the colour scale) and a unique way to interact: by swiping from bottom to up, time passes and the colour scale changes accordingly to the weather forecasts, and time of the day is showed through an animated clock icon. This is an example of how a traditional function is provided in another interactive form, supporting unexpected and maybe more pleasurable experience to check the weather forecast.

As a result our attempt here is to envision a new design perspective on smart technology. First of all to overcome the marketing and overpassed definition of smart technology as “more efficient” way to seamless accomplish daily task through automated system. This objective follows the path traced by other researches in interaction design realm, as stated in previous paragraph. In second instance, we argue to give a contribution to the evolving perspectives on experience and interaction design, finding in the root of design, the aesthetics, a way to characterized interactive artifacts making them expressive in their formal manifestation.

REFRAMING “SMART”

WHY AESTHETICS OF INTERACTION IS FUNDAMENTAL IN THE DESIGN OF SMART OBJECTS

Nowadays the adjective “smart” is widely used to characterize new products and services. This word is so diffused that it becomes very hard to identify a clear definition of the category. From a commercial point of view, the adjective is nowadays employed as a synonymous of “high tech” but, in this way, we lose the primitive meaning associated to it. Very often, the term

is associated to products and systems related to automation and efficiency.

Weiser et al (2009) defines smart environments

a physical world that is richly and invisibly interwoven with sensors, actuators, displays, and computational elements, embedded seamlessly in the everyday objects of our lives, and connected through a continuous network“.

This vision is becoming reality thanks to the diffusion of the Internet of Things (IoT). The sharing of information between objects increases their abilities and transforms their nature, so they are not anymore steady objects with a static behavior, but they can change their behavior according to the information they receive from sensors, from the web or other. For this reason, the interaction with these objects cannot be designed assuming them as static, but should instead afforded as a dialogue evolving in time.

Let’s consider a common object such as an alarm clock, and analyze some possible evolutions. A traditional alarm clock allows the user to set up the exact time (input) in order to make the alarm clock ring (output); usually, this is done through a traditional control system made of buttons and levers. In a possible evolution, an alarm clock is connected to weather forecast and traffic information. Thanks to this information, the alarm clock could, as an instance, ring earlier if traffic jam is detected on the way to the work for the user. In this case, the user sets up the time or the expected aim of being wakened -going to work- producing a first input; weather and traffic information (secondary inputs) influence the time when the alarm clock will ring (output).

With interactive solutions, many variables can influence the behavior of objects and these characteristics produces two main effects:

  • a loss of control by the user related to the amount of decision assigned to the "smart" components;
  • a evolution in time of the behavior of the objects that evolves in time according to the changing variables.

These two aspects cannot be controlled through traditional static paradigms of interaction, and they require the design of dialogues between user and object. For this reason, mere user centred approaches need to be supported (or substituted) by the ability of adding meaning to interaction through design. 

To this purpose, we consider as particularly relevant the work by Wendy Ju (2008) notably she introduces the concept of Implicit Interaction. In the interaction between a user and a product, she identifies two kinds of interactions: 

  • the explicit ones, that are evident for the user, and are the usual product of interaction design methods;
  • the implicit ones, which are more similar to the fundamental non verbal interaction between two people in their communication process, and that will be more and more important in the design of smart objects.

To sum up, we can define as smart those objects that are connected not only to the user but also to other sources of information; their behavior is driven by the user but also by other elements. Smart objects are able to react to the user actions (and are therefore reactive) but they can also take action in a self-determined manner (proactive), evolving their behavior according to the information they have and get.

The number of such smarts objects is growing very fast; apparently, it is a silent phenomenon of which we are not fully aware. According to Prof. John Barret (Cork Institute of Technology) in 2030 the number of connected object around each of us will be around 4000; furthermore, the number of internet connections and of data exchanged between objects - what we call Internet of Things - is already larger then the so called "internet of people".

As an instance, we can analyze the set of products belonging to high technological companies such as Google. Google products appearing on the market in the latest years moved out of our computers, first toward our mobile phones and tablets, later to televisions; with the acquisition of Nest, they are now entering in the home environment and they will soon enter the car sector. We think that this change poses a call for action for designers, since every aspect of our life is involved in this process; the role of design is to drive the technological evolutions of objects beyond usability and efficiency, toward maybe more complex, but meaningful experiences.

To advance in this direction, we created some empirical experiments, so to explore new methodologies and new paradigms of what we defined as aesthetics of interaction. In the following paragraphs we describe one of this experiments, and what we learned from it.

THE MAMA LAMP: FROM A SILENT LEARNING TO A CONSTRUCTIVE DIALOGUE

From the analysis of the most important case studies in the realm of smart objects, an important recent trend is represented by objects capable “to learn” and to adapt their activity to users behavior.

An example is the Nest thermostat. This object learns, automatically and silently, the user habits and it adapts by regulating the HVAC system in the house according to the users behavior. This approach is interesting since it raises the question of how we can design the learning process. We based some experiments on this inspiration so to design some concepts for a new kind of smart table lamp (figure 2).

FIGURE 2 - “MemoryLamp” video-scenario – In this supposed solution, a repetition of behaviors is learned by the smart lighting system and translated into a model of interaction. The placing of the computer on the table becomethe “switching on button”

Our smart lamp is enriched with sensors related to pressure, sounds and motion capture. In different experiments we explored different modes of interaction and dialogue between the lamp and the user.

In the first concept, the lamp captures the actions of the user switching on/off the light, observes and remembers events taking place on the table or around it, and learns the behavior of the user by recording the repetition of the same actions so to define recognizable patterns. After learning, the lamp recognizes pattern associations and it switches on or off the light when it detects a memorized behavior. This example of lamp, that we called Memory Lamp, is an extreme tentative to build an automatic and efficient smart object, based on what Hassenzahl define as “Aesthetics of Convenience” (2011). The lamp indeed tries to have a smooth and silent behavior, reducing to the minimum requirements of input form the user, with the goal to adapt the lighting function to the human behavior. In this case the design of the smart lamp was driven only by a concept of efficiency and automation, and, as a consequence, we think it created a flat experience for the user.

To change the user experience with respect to the ways the user interact with the lamp, we designed a smart lamp with a different behavior, so creating a “MamaLamp”. The main characteristic of ­the new lamp is its ability to detect people's behavior and to behave proactively in order to build an active and constructive dialogue that can evolve in time. This lamp is not functional in the traditional sense, and its behavior is not simply guided by the aim to pander a forecasted need of the user. To do so we acted on the Why level (Lenz et al., 2013) by imagining the lamp as a supportive work/study partner, able to highlight and to force positive behaviors, and to remark negative ones. More importantly we worked on the How level (Lenz et al., 2013) by working on the formal attribute of the interactive language, or, better, on the aesthetics of the interactive experience. The lamp uses the action of switching on or off the light and different levels of light intensity to communicate with the user. In some cases, as described in figure 3, the lamp does not respond to a user action, but behaves in a self determined manner, in order to enforce the meaningfulness of the communication process. The lamp reacts to this behavior of the user in a constructive way, which we believe to be the base for a dialogue with smart object.

FIGURE 3 - “MamaLamp” video-scenario – Whith the “MamaLamp”, if a) the user is taking a rest from the work, b) the lamp switches on to communicate it is time to go back to work, but in c) the user is “kicking” the lamp to switch if off again, almost trying to translate in interactive paradigms, the sentence “come on, let me rest another 5 minutes”

RESULTS AND DISCUSSION

This experiment demonstrates that the same smart lamp can have completely different meaning and impact on the life of the user according to the kind of designed behavior.

It is not enough to design the lamp in itself but it is fundamental to design the behavior of the lamp, and for doing so it is vital to consider the aesthetics aspect of it and not only the efficiency and usability parameters.

We would like to underline that this experiment represent the very early step of our research and it is meant to demonstrate the existence of the phenomenon we described. The lamp experiment is composed by two video-scenarios, that we consider to be a form of thought experiment (gedankenexperiment Einstein). We chose to work on the form of the video-scenario because it allowed us to experiment and give visual identity to the formal aspects of the interactive process, which are often implicit and difficult to represent (Vitali et al., 2014).

We chose to work on a smart lighting system for two reasons: firstly, it involves a limited number of output actions (the light can switch on/of or change intensity); secondly, even if limited and simple, the light provides a powerful emotional and metaphorical impact in the relationship with the user.

Given so we would like to underline that was never our goal to design a useful or ready to sell product, but only a space where to experiment different aesthetics aspect of the interactive dialogue, to be successively framed in a design tool to be applied later in a concrete design process.

CONCLUSION

We described the emerging phenomenon of the smart objects and we gave a definition of it.

We analyzed the actual situation of the research in the realm of interaction design, underlining the need by many researcher of an approach that overcome the user centred design approach, giving space to what we identified under the umbrella of Aesthetics of Interaction.

We believe that this different approach is even more important when we work on the design of smart objects, given the nature of the object itself.

When we design a smart object, we need to design the behavior the object will have and not only its function. Indeed we cannot design objects that have a behavior based on pure automation, usability and efficiency, because we think it will result in a poor user experience and in a rejection in the adoption of the object itself. We believe that an aesthetics research allows the designer to explore the communicative potentialities of a smart technology, as well as in the case of static products it allows to explore the formal aspects related to form, shape, colors and emotions.

To work on the formal attribute of the interactive experience allows the designer to create an evolving interactive dialogue, in respect to a one way (user to object) communication.

Acknowledgements

This work was partially supported by the Joint Open Lab “S-Cube” – Telecom Italia S.p.A. – Innovation Division.

REFERENCES

Backlund, S., Gyllenswärd, M., Gustafsson, A., Ilstedt Hjelm, S., Mazé, R., & Redström, J. (2007). Static! The aesthetics of energy in everyday things. InProceedings of Design Research Society Wonderground International Conference 2006.

Folkmann, M. N. (2010). Evaluating aesthetics in design: A phenomenological approach. Design Issues, 26(1), 40-53.

Hallnäs, L., & Redström, J. (2001). Slow technology–designing for reflection.Personal and ubiquitous computing, 5(3), 201-212.

Hassenzhal, M. (2011). Towards an aesthetics of friction, TEDx talk, find online: https://www.youtube.com/watch?v=ehWdLEXSoh8

Hassenzahl, M., & Laschke, M. Pleasurable Troublemakers. In S. Walz & S. Deterding (Eds.), The Gameful World. (2014), Cambridge, MA: MIT Press (in press).

Ju, W., & Leifer, L. (2008). The design of implicit interactions: Making interactive systems less obnoxious. Design Issues, 24(3), 72-84.

Laschke, M., Hassenzahl, M., & Diefenbach, S. (2011, June). Things with attitude: Transformational Products. In Create11 Conference (pp. 1-2).

Lenz, E., Diefenbach, S., & Hassenzahl, M. (2013). Exploring relationships between interaction attributes and experience. In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces (pp. 126-135), ACM.

Lim, Y. K., Lee, S. S., & Lee, K. Y. (2009, April). Interactivity attributes: a new way of thinking and describing interactivity. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 105-108). ACM.

Lowgren, Jonas (2013): Interaction Design - brief intro. In: Soegaard, Mads and Dam, Rikke Friis (eds.). "The Encyclopedia of Human-Computer Interaction, 2nd Ed.". Aarhus, Denmark: The Interaction Design Foundation. Available online at https://www.interaction-design.org/encyclopedia/interaction_design.html

Vitali, A.A., Spadafora, M., Pillan, M. (2014). Sketching Interactive Experience: Video Scenarios to Support Imagination and Co-design. In Proceedings of Nord Design 2014 Conference – Creating Together, 27-29 August 2014, Helsinki, edited by M. Laakso, K. Ekman, Design Society & Aalto University, ISBN 978-1-904670-58-2, ISBN 978-1-904670-59-9

Weiser, Mark, Rich Gold, and John Seely Brown. "The origins of ubiquitous computing research at PARC in the late 1980s." IBM systems journal 38.4 (1999): 693-696.

­