Towards Gesture Based Exploration of an Information Space on a Large-Screen Public Display

Abstract

This thesis aims to create a mid-air gesture-based large screen public display for browsing a hierarchical information space. Past work on interactive public displays has provided a solid grounding on challenges such as display and interactivity blindness. However ways to use mid-air gestures for interaction with public displays has been less explored. Public displays that are controlled by mid-air gestures are quite novel, even in research contexts and limited in-the-wild experimental use. Moreover the dominant work on mid-air gesture-driven public displays involved games and playful interaction. By contrast, this thesis explores how to support serious interaction to browse a hierarchical information space. This thesis presents the design, implementation and evaluation of the Media Ribbon, a gesture-based interactive large screen information display for use in a public space. We describe a series of in-the-wild studies conducted to gain an understanding of the factors affecting learning the gestures and interaction at these displays. The first prototype had a focus on reliable interaction. The next study explored aspects that affect the learnability of gestures, the influence of feedback on interaction, users’ motivation to use and play with the display and difficulties they experienced. The final study explored the effect of user representation on usability, learning the gestures and play at the Media Ribbon. This thesis makes several contributions in the field of public display research and mid-air gesture-based interaction. The first key contribution is the creation of the Media Ribbon. This includes the design of the interface that allows for browsing a hierarchical information space at a large screen public display. This involved: the design of abstract commands needed to interact with a hierarchical information space; and the design and implementation of a gesture vocabulary to enact these abstract commands. The other key contributions are the series of in-the-wild studies on the Media Ribbon exploring learnability of gestures, users’ motivation and the effect of User Interface elements on interaction. These are key to validate each of the design elements and inform the design of future displays. Our studies demonstrate that the Media Ribbon gestures are readily learnt and are usable without any prior knowledge or experience of the technology, and that the user representation has a significant impact on the social behaviours of users at these displays.