Synlab: Papers

This paper presents Tangible Tracking Table, an interactive tabletop display, and Optical Chess, a strategy game. We discuss the design and implementation of both systems and report our evaluation game play sessions with young adults, with a special focus on how the Tangible Tracking Table enhances interaction over a point-and-click interface.

We investigate the cognitive connection players create between their own bodies and the virtual bodies of their game avatars through tangible interfaces. The work is driven by experimental results showing that execution, perception and imagination of movements share a common coding in the brain, which allows people to recognize their own movements better. Based on these results, we hypothesize that players would identify and coordinate better with characters that encode their own movements. We tested this hypothesis in a series of four studies (n=20) that tracked different levels of movement perception abstraction, from own body to that of an avatar’s body controlled by the participant, to see in which situations people recognize their own movements. Results show that participants can recognize their movements even in abstracted and distorted presentations. This recognition of ‘own’ movements occurs even when people do not see themselves, but just see a puppet they controlled. We conclude that players – if equipped with the appropriate interfaces – can indeed project and decipher their own body movements in a game character.

Tangible interaction is a growing area of human–computer interaction research that has become popular in recent years. Yet designers and researchers are still trying to comprehend and clarify its nature, characteristics, and implications. One approach has been to create frameworks that help us look back at and categorize past tangible interaction systems, and look forward at the possibilities and opportunities for developing new systems. To date, a number of different frameworks have been proposed that each provide different perspectives on the tangible interaction design space, and which can guide designers of new systems in different ways. In this paper, we map the space of tangible interaction frameworks.We order existing frameworks by their general type, and by the facets of tangible interaction design they address. One of our main conclusions is that most frameworks focus predominantly on the conceptual design of tangible systems, whereas fewer frameworks abstract the knowledge gained from previous systems, and hardly any framework provides concrete steps or tools for building new tangible systems. In addition, the facets most represented in existing frameworks are those that address the interactions with or the physicality of the designed systems. Other facets, such as domain-specific technology and experience, are rare. This focus on design, interaction, and physicality is interesting, as the origins of the field are rooted in engineering methods and have only recently started to incorporate more design-inspired approaches. As such, we expected more frameworks to focus on technologies and to provide concrete building suggestions for new tangible interaction systems.

Many research efforts today explore how digitally augmented tables enable face-to-face interaction with digital content and applications. Yet the design of digital tables is still largely driven by the constraints and requirements of the underlying sensing technologies. In order to move digital tables into our real-world physical spaces, researchers need to work closely with architects and industrial designers in order to engage the knowledge and skills from a long history of physical design and fabrication in the creation of tabletop systems. This paper presents Architales, an interactive story table for gallery exhibition, developed as an experiment in physical/digital co-design. We describe the creation of the interactive table, tangible storytelling system, and story content, which evolved together in a closely unified design process, drawing on the skills from computing, media, design and architecture. We also describe lessons learned about the interdisciplinary design process and the creation of physical/digital artworks.

Computationally-enhanced tabletops featuring tangible objects for participant interaction hold promise for the creation of new forms of media, including documentaries and stories constructed by individuals or small groups in distributed or face-to-face settings. Many computer-aided storytelling systems feature either algorithms to arrange story elements automatically or story structures to guide authors in presenting their tales. In contrast, the KinoPuzzle system affords a high degree of freedom for authors in constructing tabletop experiences and for participants in exploring multi-viewpoint narratives. This format for story-telling combines the representational depth and flexibility of the digital database with the expressive power of the collage surface, offering advantages in terms of open-ended dialogic juxtapositions and the collaborative exploration of reality-based material.

RENATI is an acronym for recontextualizing narratives for tangible interfaces. It serves as an umbrella term for our art/research experiments within a hybrid environment that uses oral narratives, and non-generative and immersive art with sensing technologies to create tangible narratives. In this paper we introduce our first prototype, which uses a custom-built mannequin to allow viewers to engage with a multi-viewpoint story titled Flying Over Purgatory.

RENATI is an acronym for recontextualizing narratives for tangible interfaces. It serves as an umbrella term for our artistic experiments within a hybrid environment that uses various forms of narratives, such as oral narratives, diaries, essays, screenplays, and non-generative and immersive art with sensing technologies, to create tangible narratives.

The International Conference Tangible and Embedded Interaction (TEI) is the first conference with "tangible" in its title, motivated by the fieldâ€™s growth over the past decade. One of this research area's most fascinating features is the range of perspectives and disciplines, including HCI, the arts, design, technology, and architecture. TEI invites a multi- and interdisciplinary community to submit contributions from their specific viewpoints and approaches. It brings together artists who employ tangible media with computer engineers developing toolkits, physical appliance designers with researchers conducting user studies, and researchers exploring novel interface technologies with interaction designers who develop mixed-media systems.

WiiArts is an experimental video, audio and image processing art project that makes use of pre-existing sensing technologies provided by Nintendo WiiRemotes and a Sensor Bar. Currently, most WiiRemote-based physical interactions have been designed to mimic the gesture of body movement in sports and other action-based games. These Wii games are generally competitive in nature, and players interact by responding to predefined interaction rules in either a single-user or multi-user mode. Making use of the WiiRemote as a pre-existing tangible and embedded interface, we explore applications that can engage participants in active and expressive art creation in a collaborative manner. In this paper, we describe several prototype applications based on this concept: Illumination (draWiing), Beneath (Waldo), WiiBand, Time Ripples. In these applications, three interactors can work together to compose both images and sounds.

In screen-based experiences, the screen itself can become the physical device used for interaction. The "move-ability" of the screen affords interactivity between the screen artifact and the viewer and between the virtual space and the physical space. As a movable screen interface, I have created the spinning screen, a two-sided monitor mounted on a revolving base. This interface invites user interaction through the action of spinning the screen. User interactions with the spinning screen can support diverse temporal and spatial responses, thereby enriching usersâ€™ experiences. The spinning screen enables the viewer to grasp the interplay between visibility and invisibility, creating an aesthetic experience. The angle and direction of rotation affects the displayed visuals and audio output.

In screen-based experiences, the screen itself can become an interactive element. The "move-ability" of the screen affords interactivity between the screen artifact and the viewer and between the virtual space and the physical space. Cross-Being: Dancer features a movable screen interface, a spinning screen based on a two-sided monitor mounted on a revolving base.

The ontological condition of the screen is "in-between" several boundaries: the boundary between space and time, virtual and real, and immersion and rejection in the sense of similarity and difference. By re-categorizing, connecting and blurring these existential boundaries around the screen, the approaches described here form two kinds of screen experiences: three-dimensional screens in video installations, and 'movable' screens in real-time video and sound installations.

Seeing multiple moons visible overhead is not our normal experience in the real world. Even though we can picture this kind of experience in our imaginations, we cannot properly share it with others. It can only visually exist in our own individual minds. We would like to share this experience by creating a poetic installation space of virtual and real, where multiple moons can follow us around. In this gallery room, the audience can share their own imaginative and individual experience with others.

Flora Electronica is a collaborative artwork created by artists and researchers during a course in tangible media arts. This playful artificial/computational garden is a stage where spontaneous media performances unfold, manifested across both the physical and virtual worlds. The emotional tone of the space is set by the real-time perceptions/interactions of a participating audience, both local and remote.

The TViews Table Role-Playing Game (TTRPG) is a digital tabletop role-playing game that runs on the TViews table, bridging the separate worlds of traditional role-playing games with the growing area of massively multiplayer online role-playing games. The TViews table is an interactive tabletop media platform that can track the location of multiple tagged objects in real-time as they are moved around its surface, providing a simultaneous and coincident graphical display. In this paper we present the implementation of the first version of TTRPG, with a content set based on the traditional Dungeons & Dragons rule-set. We also discuss the results of a user study that used TTRPG to explore the possible social context of digital tabletop role-playing games.

Many research efforts today explore how digitally augmented tables enable face-to-face interaction with digital content and applications. Yet the design of digital tables is still largely driven by the constraints and requirements of the underlying sensing technologies and digital systems. In order to move digital tables into real-world physical spaces, researchers need to work closely with architects and industrial designers in order to engage the knowledge and skills from a long history of physical design and fabrication. Architales is an interactive story table for gallery exhibition developed as an experiment in the physical/digital co-design of the physical table and environment with the digital story system and content.

In this paper we describe the concept, design, and implementation of Murmur, an interactive kinetic display made of one hundred computer CPU fans. Murmur responds to sound input from its environment via embedded microphones to produce patterns on a reactive surface. The reactive surface consists of hinged paper pieces situated in front of each fan. When activated by sonic elements in the environment, including sounds intentionally generated by an interactor, Murmur responds by turning on and off its fans in a sequence. The wind pressure generated by the movement of the fans stimulates the surface, forcing the paper up and out to create a variety of dynamic patterns. Each pattern represents characteristics of the sonic environment. We also analyze the feedback received from the interactors and discuss the possible ways of making the interaction more immersive.

We developed Tangible Tracking Table (TTT), an interactive tabletop display that allows both fingertip and tangible object tracking. Multiple users can simultaneously control a variety of tagged objects that can perform different tasks. We also introduce several applications on TTT that utilize the advantages of this tangible user interface.

The Graphical User Interface (GUI) has created an efficient work environment for many applications. However, when users are confined by keyboards and mice, they lose the ability to interact with the virtual world using habits from the real world. Our research examine how emerging modes of authorship, such as wikis, can be used to generate new possibilities for bringing atoms and bits together for digital annotation. Our goal is to combine the everyday habits in reading books with emerging digital possibilities. In this paper, we present a prototype system called WikiTUI, which brings digital media to physical paper books. This system allows readers to access the digital world through fingertip interactions on books, and enables them to share information with other readers using wiki technology. WikiTUI not only bridges the gap between the digital and the physical worlds, but also facilitates multiple contributions to a reference base spanning across these worlds. We present user evaluations of the WikiTUI prototype and discuss research implications.

The workshop aims to bring together researchers and practitioners working on subjects related to digital games with tangible interaction. We would like to involve participants with backgrounds in academia as well as industry, from diverse fields such as HCI, computer science, edutainment, interaction design and game design. Some of the topics we plan to address during the workshop are: different tangible interaction styles, designing for specific game types, and the advantages and disadvantages of different sensing and object tracking technologies.

In this paper, we describe the design and implementation of Teaching Table, an interactive tabletop audio-visual device aimed at enhancing the learning experience for prekindergarten children by involving them in physical activities. Using electromagnetic sensing technology, the table can track tagged objects placed on its surface, accurately identifying their type and location while providing a coincident visual display and audio feedback. Teaching activities that are aimed at developing early math skills have been created for the table in alignment with standard curriculum guidelines for pre-K schools. Additionally, we include software based assessment tools for mentors/teachers to easily track an individual child's progress during the process of interacting with the table.

In screen-based experiences, the screen itself can become the physical device used for interaction. The "move-ability" of the screen affords interactivity between the screen artifact and the viewer, and between the virtual and physical spaces. We have created a movable screen interface, called the tilting table, which provides a display surface via overhead projection. This interface invites user interaction through the action of tilting the tabletop. The degree of tilt affects the displayed visuals and audio output. This simple interaction makes users feel they have a closer connection to the virtual imagery, and the screen thus blurs the boundary between them. In this paper, we introduce four applications that have been implemented for the tilting table. Two use an artistic approach to create expressive and entertaining media interactions, while two have been developed for navigating large information spaces. Each application is described and user feedback is discussed.

In screen-based experiences, the screen itself can become the physical device used for interaction. The "move-ability" of the screen affords interactivity between the screen artifact and the viewer and between the virtual and physical spaces. As a movable screen interface, we have created the spinning screen, a two-sided monitor on a revolving base. This interface invites user interaction through the action of spinning the screen to manipulate media content. By allowing viewers to grasp the interplay between visibility and invisibility, the spinning screen can enrich user experiences of virtual imagery through direct tangible interaction in both playful and practical contexts. We describe an art piece and an application for 3D visualization and modeling.

The TViews Table Role-Playing Game (TTRPG) project explores the creation of digital tabletop role-playing games on the TViews table, in order to bridge the separate worlds of traditional role-playing games with the growing area of massively multiplayer online role-playing games. The TViews table is an interactive tabletop media platform that can track the location of multiple tagged objects in realtime as they are moved around its surface, providing a simultaneous and coincident graphical display. In this paper we present the implementation of the first version of TTRPG, with a content set based on the traditional Dungeons & Dragons rule-set. We also discuss the results of a user study that used TTRPG to explore the possible social context of digital tabletop role-playing games.

Emergent game formats, such as machinima, that use game worlds as expressive 3D performance spaces have new expressive powers with an increase of the quality of their underlying graphic and animation systems. Nevertheless, they still lack intuitive control mechanisms. Set direction and acting are limited by tools that were designed to create and play video games rather than produce expressive performance pieces. These tools do a poor job of capturing the performative expression that characterizes more mature media such as film. Tangible interfaces can help open up the game systems for more intuitive character control needed for a greater level of expression in the digital real-time world. The TUI3D project (Tangible User Interfaces for Real-Time 3D) addresses production and performative challenges involved in creating machinima through the development of tangible interfaces for controlling 3D virtual actors and environments in real-time. In this paper, we present the design and implementation of a tangible puppet prototype for virtual character control in the Unreal game engine and discuss initial user feedback.

The past several years of computer interaction research have shown an increasing interest in tabletops for shared user interactions through touch or tangible objects. Digital media tables offer the potential to expand our digital interactions into casual social settings that are not appropriate for desktop platforms, such as home living rooms. We have developed a tangible media table called TViews, which provides an extensible architecture to enable multiuser interactions with a range of media applications and content via tagged tangible objects. The TViews object positioning utility functions on the surface of an embedded display and enables real-time tracking of a virtually unlimited set of uniquely identified wireless objects that can be used on the surface of any similar table. These objects can be physically customized in order to suit particular applications, and can provide additional functionality through external input and output elements on the objects themselves. In this paper, we present a first field trial of TViews to gain some initial insight into how such a device could be adopted in a real-world home.

With the rapid and continuous growth of digital media entertainment, we need new kinds of platforms and technologies that can better bridge our physical and digital worlds. Rather than confining our interactions to single-user portals into the digital space, emerging platforms such as digital tabletops can enable face-to-face social interaction with digital games. We present here a new sensing architecture for creating tangible media tables. The TViews architecture provides tangible interaction through real-time tracking of multiple objects on a sensing surface equipped with an embedded display. The design can generalize across diverse applications and physical contexts, ranging from home living rooms, to classrooms or public settings. We describe the technical design and our initial game development for the table.

Radio Frequency IDentification (RFID) technology has recently been moving into everyday use contexts. Previous work has shown wearable RFID systems to be a viable mechanism for collecting data about a userâ€™s interactions with her environment. In this paper, we present wearable RFID systems as a promising new direction in tangible game interfaces. We provide an overview of the affordances of RFID for game-play, and present some existing and future wearable RFID-based games. We also describe the construction of a cheap, easy-to-build wearable RFID system and present a how-to resource for other researchers interested in building off our work.

We present a hybrid positioning and communication system for tracking interaction objects called 'pucks' on the surface of a large LCD or plasma display. Pucks are smart sensor packages consisting of a microcontroller as well as a contact-type acoustic receiving transducer and an infrared or radio data link. A puck may take the form of a standalone interaction object, or the puck circuitry may be integrated into an existing object, such as a digital camera, cellphone, PDA, or other device. In this work we take advantage of the glass surface atop an LCD or plasma display as a communication and sensing medium, and launch 200KHz Gaussian-shaped acoustic ranging pulses into that medium from transmitting transducers adhered to the corners of the glass. We present experimental results demonstrating millimeter-scale puck positioning accuracy over the entire surface of a 32-inch LCD, at an update rate of 100Hz. We also demonstrate the scalability of this approach to much larger displays. In our first implementation, power consumption of each puck is 3V at 12mA during data transmission, 5mA during positioning, and 50Î¼A when idle, yielding 6-8 hours of continuous tracking from a 90mAH prismatic lithium polymer battery.

Tangible Comics is a computer vision based full-body interactive storytelling environment that also functions as a comics generator. Prevailing applications of full-body computer vision have not utilized the full storytelling or performance potential of these environments. Our aim is to produce an environment that can create a space for redefining the conventions of comics, performance, film, photography, and animation. In relation to that, we are exploring the design problems that can arise when computer vision technology is contextualized in an interactive story telling environment.

The Shadow Box is a tangible computing project that exploits visual association and auditory clues to teach children the representational relationship between words and their meanings. The Shadow Box contains three major components: the main box, picture blocks and word blocks. The Shadow Box activates when a block or a matching pair of blocks is placed inside. The box prompts children to find matching blocks and combine them together. When children successfully combine the right word and picture, the box rewards them with an animated video as if they had made the objects come alive. An informal study shows that children responded positively to the concept of the box. They played with it for a length of time and engaged in a collaborative learning process with other children.

WikiTUI is a tangible multimedia annotation system that allows users to manipulate multimedia information in common physical books. WikiTUI also facilitates the exchange of information with other readers through wiki services. Using tangible interface techniques, we seek to bring the collaborative annotation and authoring capabilities that are supported by the growing space of online wikimedia into the real world, thereby extending the use of paper books into new realms.

This article describes the design of WikiTUI, a tangible user interface system that allows readers to manipulate multimedia information tied to page-level locations in common books. This system will also facilitate the exchange of information with other readers using wiki technology. While other projects have coupled the dual worlds of atoms and bits, few have combined the idea of collaborative and distributed authoring with tangible interfaces. WikiTUI not only bridges the real and digital worlds, but facilitates multiple contributions to a reference base spanning across these worlds. The goal of our work is to apply the paradigm of coupling atoms and bits in order to extend the collaborative annotation and authoring possibilities of wiki media.

In this project, I explore how to keep the interaction and learning for the pre-K students in the real world using physical objects while harnessing the power of the virtual one using Tangible Computing. In that attempt, I present here the design and implementation of "Teaching Table" - a system of digital manipulatives and display surface embedded with sensing technology and linked to computational intelligence. The table can track tagged objects placed on its surface, accurately identifying their type and location while providing a coincident visual display and audio feedback, using which activities are created that impart math knowledge to the children playing on the table. I also propose how the table can be an integral addition to the present classroom environment as it can enhance learning experiences of children in an interactive and playful way by involving them in physical activities. Focusing on the age group of pre-kindergarten children, the Teaching Table activities will attempt to develop fundamental math skills by leveraging the already existing curriculum guidelines for pre-K schools set by the State of Georgia (or, any other state as appropriate).

We are creating unified programming tools that allow developers to easily build applications for many different tangible platforms, and that can accommodate the continued evolution of the underlying sensing technologies. The Synlab API extends across several media table object-tracking platforms, RFID tagged object interactions, manipulable displays and multi-touch surfaces.

TViews is an extensible method and acoustic-based sensing framework for creating digital media tables that overcome many limitations of singlepurpose systems. The system provides a means for real-time multiobject tracking on the tabletop and for the management of large numbers of objects and applications across multiple platform instances.

Traces is an ambient installation that uses the knowledge of the intersection of bodies with floors and walls in order to represent the absence of those bodies. By acting, users explore the traces of their past actions and their absence from those actions. The technique used is based on visual hull reconstruction from multiple cameras on the GPU.

Tangible Comics is an attempt to redefine comicsâ€™ conventions for a narrative performance in a digital environment. It seeks ways of taking digital comics out of the computer screen and merging comicsâ€™ conventions into three-dimensional real space. The installation will utilize the qualities of digital games, interactive narration, digital art, performance, and augmented reality.