Tagging with Movement
Somatic Strategies for Digital Image Classification
My doctoral research investigated embodied user experience with the goal of incorporating additional sensory modalities into human-computer interaction. As computer use expands into new social and physical environments new approaches to interaction are not just possible, but necessary. These approaches require a re-conceptualization of user experience, one that takes into account embodied experience and fully integrates the moving body into models of interaction. Current Human-Computer Interaction (HCI) practices primarily consider movement as a functional component of interaction; however, movement also has a significant experiential component that has been largely ignored. My work integrates these two characteristics of movement in order to explore new possibilities for interaction, particularly in settings where the use of traditional input and output methods are impractical. My research differs from most ongoing research investigating movement interaction by focusing on the subjective human experience of movement rather than on the characteristics of observable movement which can be sensed through technology. This reversal puts the emphasis back on the human interactor and will inform the development of people-centered sensing technology.
My thesis specifically investigates the use of kinesthetic “tags” as a way to understand the meaning of movement for an individual user. Kinesthetic tagging offers an alternative to the current linguistic meta-data used to describe and classify digital objects, providing a method that supports embodied and tacit knowledge. My outcomes include an exemplar application that demonstrates the use of kinesthetic movement to tag, categorize and search for digital photographs as well as an initial framework for incorporating experiential movement into computational interaction. This particular application exploring kinesthetic movement tagging is a first step in a larger research program aimed at uncovering the features of subjective movement experience that are relevant to the design of tangible and ubiquitous computer systems.
This research incorporates participatory design methods employed in a workshop setting. In these workshops I collaborate with movement experts, primarily dancers, to explore the enactment of image attributes through movement. The participants are initially asked to sort images into meaningful categories which they then perform as short movement sequences. Throughout the remainder of the workshops additional enactments of more detailed image attributes are performed and the movements are distilled into the most basic form possible. Over the entire series of workshops correlations are identified between particular movement qualities and specific image attributes. These movements types and their associated qualities will be used to develop an exemplary application that facilitates the searching and sorting of an image collection using movement.
My research is informed by movement-based research within the field of HCI as well as by practices from art-based disciplines such as dance, theater, cinema, interactive narrative, and music. I incorporate theories of embodiment from a variety of disciplines ranging from media-art and design to cognitive science. My research methods include body-based practices borrowed from the field of Somatics as an integral component of my research practice. Somatic methods, based on direct first-person observation of the body, highlight the phenomenological aspects of experience for an individual and can be used to enhance awareness of movement. Through the use of participatory design workshops that focus on somatic awareness I am clarifying a notion of user experience that focuses on the holistic and embodied experience of being human.
A downloadable copy of my dissertation is available here.