Room-scale interactive and context-aware sensing

Number: 10353526
Published: 2019-07-16
Filed: 2018-04-20
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Sample; Alanson, Yang; Chouchang, Zhang; Yang
CPC: G06F3/044; G06F3/0446; H04W4/33; G06F3/04166; H04W4/80; G06F3/04162; G06F3/011; C09D5/24
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Room-scale interactive context-aware sensing (Sample, novel sensing).

Abstract

Human environments are typified by walls—homes, offices, schools, museums, hospitals and pretty much every indoor context one can imagine has walls. In many cases, they make up a majority of readily accessible indoor surface area, and yet they are static—their primary function is to be a wall, separating spaces and hiding infrastructure. We present the Wall++ system, a low-cost sensing approach that allows walls to become a smart infrastructure. Instead of merely separating spaces, walls can now enhance rooms with sensing and interactivity. Our wall treatment and sensing hardware can track users' touch and gestures, as well as estimate body pose if they are close. By capturing airborne electromagnetic noise, we can also detect what appliances are active and where they are located. Through a series of evaluations, we demonstrate the Wall++ system can enable robust room-scale interactive and context-aware applications.

Background

BACKGROUND(1) Below we discuss three key areas of related work. First, we discuss prior work that enables room-scale touch tracking. We then review room-scale approaches for tracking user location and pose. We conclude with systems able to detect and track objects. In particular, we focus primarily on systems that are deployed in the environment, as opposed to those that are carried (e.g., smartphones, wearables).(2) Room-Scale Touch(3) Most previous systems have achieved wall-scale touch sensing through optical approaches. For example, LaserWall used a scanned laser rangefinder operating parallel to a wall's surface to detect hand touches. Infrared emitter-detector arrays have also been used to create large interactive surfaces. Most popular are camera-based approaches, including invisible light, depth, and even thermal imaging.(4) People have also explored acoustic touch sensing approaches, for example, by attaching microphones to the corners of a desired interactive surface and using time difference of arrival methods. It is also possible to use an array of centrally located acoustic sensors for estimating the location of tap events. Researchers have also forgone absolute spatial tracking, and instead built interactions around gesture vocabularies.(5) More relevant to the systems disclosed herein are systems that use capacitive sensing. Early work by Smith et al. demonstrated a capacitive sensing wall able to detect user gestures such as swipes, though users had to stand o

Claims

1. A multimode sensing system installed on a wall of a building structure, the system comprising: an array of conductive electrodes installed on the wall, the array arranged into rows and columns of conductive electrodes; a plurality of conductive traces installed on the wall, each trace interconnecting each of the conductive electrodes in a given row or column; a transmitter electrically coupled to one of the rows or the columns of the array of conductive electrodes; a receiver electrically coupled to the other of the rows or the columns of the array of conductive electrodes; and a processor coupled to the transmitter and the receiver to perform mutual capacitive sensing to determine the position on the array where a change in capacitance is sensed, and coupled to the receiver to perform electromagnetic sensing to sense what types of electronic objects are proximate to the wall and to determine the position on the array where the electronic objects are sensed. | 16. A sensing system, comprising: a wall extending in a vertical plane, the wall being at least one meter wide and at least one meter tall; a patterned array of electrodes applied to the wall, the pattern extending at least one meter wide in a horizontal direction and the pattern extending at least one meter tall in a vertical direction; and a controller electrically connected to the patterned array of electrodes that utilizes passive electromagnetic sensing to sense electromagnetic energy from a source of electromagnetic energy, determine a location on the wall closest to the source of electromagnetic energy, and determine a type of electronic device that is the source of the electromagnetic energy, wherein the determination of the type of electronic device that is the source of electromagnetic energy is made by comparing a frequency spectrum of the sensed electromagnetic energy to known frequency spectrums of electromagnetic energy from known electronic devices. | 22. A sensing system, comprising: a wall extending in a vertical plane, the wall being at least one meter wide and at least one meter tall; a patterned array of electrodes applied to the wall, the pattern extending at least one meter wide in a horizontal direction and the pattern extending at least one meter tall in a vertical direction; and a controller electrically connected to the patterned array of electrodes that utilizes mutual capacitive sensing to sense contact by a human with the wall and the location on the wall of the human contact and that utilizes passive electromagnetic sensing to sense electromagnetic energy from a source of electromagnetic energy, determine a location on the wall closest to the source of electromagnetic energy, and determine a type of electronic device that is the source of the electromagnetic energy, and wherein the determination of the type of electronic device that is the source of electromagnetic energy is made by comparing a frequency spectrum of the sensed electromagnetic energy to known frequency spectrums of electromagnetic energy from known electronic devices.