ABSTRACT We investigate the cognitive connection players create between their own bodies and the virtual bodies of their game avatars through tangible interfaces. The work is driven by experimental results showing that execution, perception and imagination of movements share a common coding in the brain, which allows people to recognize their own movements better. Based on these results, we hypothesize that players would identify and coordinate better with characters that encode their own movements. We tested this hypothesis in a series of four studies (n=20) that tracked different levels of movement perception abstraction, from own body to that of an avatar’s body controlled by the participant, to see in which situations people recognize their own movements. Results show that participants can recognize their movements even in abstracted and distorted presentations. This recognition of ‘own’ movements occurs even when people do not see themselves, but just see a puppet they controlled. We conclude that players – if equipped with the appropriate interfaces – can indeed project and decipher their own body movements in a game character.

A growing body of work shows that compatible actions executed in parallel with cognitive tasks contribute beneficially to cognition, compared to incompatible actions. We investigate how such complementary actions are generated. Two models from imitation research, Associated Sequence Learning (ASL) and Active Intermodal Matching (AIM), were extended to develop models of complementary action generation. ASL postulates a general generation process based on learning, whereas AIM postulates a specialist process. Using a mental rotation task where participants tended to spontaneously generate parallel actions, we conducted two experiments to test the predictions of the extended models. Surprisingly, the results show that when compared to no-actions, complementary actions do not improve accuracy. The two experiments do not provide clear validation for either model of generation, but there is more support for the generalist model than the specialist one. We propose a revision to the generalist model based on this trend.

Emergent game formats, such as machinima, that use game worlds as expressive 3D performance spaces have new expressive powers with an increase of the quality of their underlying graphic and animation systems. Nevertheless, they still lack intuitive control mechanisms. Set direction and acting are limited by tools that were designed to create and play video games rather than produce expressive performance pieces. These tools do a poor job of capturing the performative expression that characterizes more mature media such as film. Tangible interfaces can help open up the game systems for more intuitive character control needed for a greater level of expression in the digital real-time world. The TUI3D project (Tangible User Interfaces for Real-Time 3D) addresses production and performative challenges involved in creating machinima through the development of tangible interfaces for controlling 3D virtual actors and environments in real-time. In this paper, we present the design and implementation of a tangible puppet prototype for virtual character control in the Unreal game engine and discuss initial user feedback