Designing for Gesture and Tangible Interaction
Mary Lou Maher and Lina Lee
ISBN: 9781627056847 print
ISBN: 9781627058865 ebook
DOI 10.2200/S00758ED1V01Y201702HCI036
A Publication in the Morgan & Claypool Publishers series
SYNTHESIS LECTURES ON HUMAN-CENTERED INFORMATICS, #36
Series Editors: John M. Carroll, Penn State University
Series ISSN: 1946-7680 Print 1946-7699 Electronic
Designing for Gesture and Tangible Interaction
Mary Lou Maher and Lina Lee
The University of North Carolina at Charlotte
SYNTHESIS LECTURES ON HUMAN-CENTERED INFORMATICS #36
ABSTRACT
Interactive technology is increasingly integrated with physical objects that do not have a traditional keyboard and mouse style of interaction, and many do not even have a display. These objects require new approaches to interaction design, referred to as post-WIMP (Windows, Icons, Menus, and Pointer) or as embodied interaction design.
This book provides an overview of the design opportunities and issues associated with two embodied interaction modalities that allow us to leave the traditional keyboard behind: tangible and gesture interaction. We explore the issues in designing for this new age of interaction by highlighting the significance and contexts for these modalities. We explore the design of tangible interaction with a reconceptualization of the traditional keyboard as a Tangible Keyboard, and the design of interactive three-dimensional (3D) models as Tangible Models. We explore the design of gesture interaction through the design of gesture-base commands for a walk-up-and-use information display, and through the design of a gesture-based dialogue for the willful marionette. We conclude with design principles for tangible and gesture interaction and a call for research on the cognitive effects of these modalities.
KEYWORDS
embodied interaction, gesture interaction, tangible interaction, interaction design, cognitive effects
Contents
2.1 What is a Tangible Interaction?
2.2 Why is Tangible Interaction Interesting?
2.2.1 Space-Multiplexed Input and Output
2.2.4 Spatially Aware Computational Devices
2.2.5 Spatial Reconfigurability of Devices
2.3 What Are Key Design Issues for Tangible User Interfaces?
2.3.1 Designing Interplay of Virtual and Physical
2.3.2 Selecting from Multiple Gestures and Actions
2.3.3 Crossing Disciplinary Boundaries
2.3.4 The Lack of Standard Input and Output Interaction Models
3 Designing for Tangible Interaction
3.1 Designing the Tangible Keyboard
3.1.1 Context of Use
3.1.2 Design Goals
3.1.3 Technology Selection
3.1.4 Design Methods
3.1.5 Interaction Model
3.1.6 Design Heuristics
3.2 Designing Tangible Models
3.2.1 Context of Use
3.2.2 Design Goals
3.2.3 Technology Selection
3.2.4 Design Methods
3.2.5 Interaction Model
3.2.6 Design Guidelines
4.1 What Is a Gesture-based Interaction?
4.1.1 Gesture Commands: Walk-Up-and-Use Information Display
4.1.2 Gesture dialogue: The Willfull Marionette
4.2 Why is Gesture-based Interaction Interesting?
4.2.1 Public Walk-Up-and-Use Displays or Objects
4.2.2 Remote Actuators, Sensors, and Displays
4.2.3 Sociable Multiple-User Interaction
4.2.4 Gesture Communication
4.3 What Are Key Design Issues for Gesture-based Interaction?