This paper proposes a framework for developing and accessing distributed multiplayer motion-based interactive applications. The framework provides an API for programmers to easily access users' motion data through a Kinect sensor, connect to a network of stations with similar features, and share the motion data. Using this framework, developers can create multiplayer motion-based games or other apps, and share them. The framework allows users to select both apps to play and the stations with which to play. The location-oriented nature of the framework (choosing station vs. user) makes it particularly useful for public installations. We demonstrate the functionalities of our proposed framework with a sample game.

Smart phones have become the hub of people lives due to the overwhelming number and extensive range of apps available in app stores that are available to support their daily tasks. On average, smart phone users have around 100 apps installed on their devices and the number is ever growing. Thus, it becomes crucial to make sure they can quickly access these apps. In this paper, we present an exploratory study to understand users' memorability of their self-defined gestures for 15 frequently used mobile apps. The results show that although participants recalled their self-defined gestures most of the time, there are still certain factors that can influence their recall. The paper further analyses the underlining reasons and discusses how such issues could be addressed from a technical perspective.

We developed the prototype application based on the observational study for the user's manipulation of a carbon paper in the real world.

Urban planning, and campus planning in particular, can benefit greatly from technology that fosters and leverages collaborative, participatory planning which includes all affected stakeholders. Interactive surfaces such as tabletops and walls, and spaces such as 3D environments are examples of such technologies. Our research aims to explore and assess the appropriateness and effectiveness of such technologies in participatory campus planning. We present here a prototype system that combines the tangible and multitouch surfaces with a virtual world platform to provide a participatory planning approach for campus development projects.

Isolation and loneliness has been shown to be a significant cause of disease and mortality among elderly adults. The coproduction of activity is one potential method of prevention, through the promotion of social interaction and community engagement. The present study aims to facilitate coproduction in a continuing care retirement community, promoting health and wellness in elderly adults. To achieve this, we engage in the participatory design of a public interactive display for organizing community activity. We explore design needs and usability concerns from elderly residents, and discuss the trajectory of co-design work to realize this technological intervention.

We present techniques for exploring time-dependent scientific data using spatially aware mobiles and large displays, including techniques for specifying arbitrary 2D views and bookmarking.

Tactile devices with ultrasonic vibrations (based on squeeze film effect) using piezoelectric actuators are one of the existing haptic feedback technologies. In this study we have performed two psychophysical experiments on an ultrasonic haptic tablet, in order to find the minimum size of a tactile element on which all the users are able to perfectly identify different types of textures. Our results show that the spatial resolution of the tactile element on haptic touchscreen actually varies, depending on the number and types of tactile feedback information. A first experiment exhibits three different tactile textures, chosen as being easily recognized by users. We use these textures in a second experiment, and evaluate minimal spatial area on which the chosen set of textures can be recognized. Among other, we find the minimal size depends on the texture nature.

Foldable3D: Interacting with 3D Content Using Dual-Display Devices

Fun-f! Game-furniture experimenting interactive surfaces for children's sensory learning.

In this paper, we explore the use of multimodal interfaces (tangible and gestural) in collaborative gameplay. In our setup, gestural interaction is performed to perform tasks with reference content on a wall display, in support of a main activity involving tangible interaction on a tabletop display. We designed two games using this configuration, in order to explore how children share these multimodal tasks during collaborative gameplay. We conducted a pilot within-subjects user study, piloting the games with 35 children between the ages of 8 and 15, and considered the impact of a number of factors (group size, age range, game type and interaction style) on gameplay. We present lessons learned from prototyping and pilot testing tangible+gestural multi-display systems, and describe the resulting refinements made in our design and implementation.

Data science is an interdisciplinary field at the intersection of computer science, mathematics, and subject matter expertise with the purpose to gain insights from relatively large sets of data. With elementary school age children as the target audience, we developed an educational collaborative game, Habitat Explorer for the MultiTaction display to introduce users to the core data science cycle components of data collection, exploration, and visualization. Users capture "sea creatures" in a collection jar, sort their collection into a data table, and then are able to create a bar chart visualization using their data. These components are interspersed with questions assessing user comprehension of the simple biological and data science concepts introduced during the game. The main goal of this research is to understand the potential of multi-touch displays to facilitate data science education with future aims to develop more complicated data exploration applications.

A multidisciplinary team of scientists is shown interactively exploring a 2048^3 voxel scan of a primate tooth on a large-scale display with an interaction client running on a laptop.

A child participant inputting text with the color-coded autocorrect method.

Lightweight Capacitance-based Block System for 3D Space Interaction.

Workers in factories often have to stop an operation to confirm various assembly instructions, for example, component numbers and/or the location to place a component, this is particularly the case with exceptional or inexperienced operations in a mixed-flow production line. These types of operation interruptions are one of the most significant factors linked to a decreasing productivity rate. In this study, we propose a novel method that estimates, in real time, the pose of a manufactured product on a production line without any augmented reality (AR) markers. This system projects instructions and/or component positions to help a worker process production information quickly. In this paper, we produced assembly-support system experimentally using projection-based AR. We develop a highly accurate object pose estimation method for manufactured products. The result of this experimental evaluation indicates that the combination of ORB and our algorithm can detect an object's pose more precisely than ORB only. We also develop an algorithm that is robust even if a part of an object is occluded by a worker's hand. We consider that this system helps workers understand instructions and component positions without the need to stop and confirm assembly instructions, thus enabling more efficient operation of tasks.

This work analyzes challenges of business data presentations in board meetings regarding presentation control, visualizations, audience discussions, and remote presence. We propose ideas tackling those challenges using HCI principles.

Sports Support System can track users and ball movement, and provides the user the sense of high-level player by the interactive visualization of those trajectory and speed on the field.

Optical sensing technologies are among the most versatile hardware solutions for interactive surfaces, as they are capable of recognizing touch as well as (limited) hover state in addition to printed tokens. One widely used system is the Pixelsense/SUR40, currently one of very few devices which provides these capabilities in the form factor of a regular table, thereby allowing working at the device in a sitting position. Unfortunately, the device has been discontinued by the manufacturer, provides only an unsupported SDK on an outdated operating system, and has a gathered a reputation for high latency as well as sensitivity to environment light. In this paper, we present our research into modernizing and extending the SUR40 system. By switching to a Linux operating system running a custom video driver, we are able to provide lower latency, support other types of optical tags and improve the system's robustness, particularly regarding external lighting conditions. We present an analysis of the device's internals, a comparison of quantitative performance measurements, and an outlook into extending the tangible interaction capabilities with an improved cross-platform development framework.

Synchronized data-conferencing between two remotely located large-scale displays. Small groups can engage in mixed presence collaboration using our techniques of data-duplication or advanced data-synchronization options.

We have built a display wall system with four liquid crystal displays to provide a very large visual area for university lectures. A program was developed to transform the display wall into an electronic blackboard. It has been used in real classroom practice. Based on the research results, we demonstrate the benefits of a digital multi-display system for classroom learning.

Our new touch detection technique which utilizes finger's shadows detects a touch even on non-flat surface.

Twosome Coloring on a dual-sided transparent surface

Users using WebTEM on different devices.