Interacting through eyes

ABSTRACT: Advanced interaction devices are needed to improve the communication between robots and humans. In this paper we regard computers as specialized robots, provided with a number of sensors for understanding the environment in which they operate (humans included), and virtual actuators, which cannot transport materials but can transport information in the form of sensory feedback. Given such a theoretical framework, we focus our attention on the design of a machine vision system for the understanding of gaze pointing actions in advanced interaction applications. Our approach, which is based on the combination of eye tracking in images and a simple drag and click semantics, allows the definition of an ocular mouse for interacting with on-screen scenarios. Experimental results obtained with our system are provided, which on the one hand demonstrate the validity and extensibility of our approach to different situations, and on the other point out the fundamental role played by sensory feedback and human adaptation characteristics in man-machine interaction.