CONTROL SYSTEM USING AESTHETICALLY GUIDED GESTURE RECOGNITION

Number: 20170285757
Published: 2017-10-05
Filed: 2016-03-31
Assignee: DISNEY ENTERPRISES, INC.
Inventors: ROBERTSON; JAMES R., SCANLON; RAYMOND J., HONECK; MICHAEL R., JAY; ANGELA, ROUSE; CORY J., IRMLER; HOLGER
CPC: G06F3/014; G06F3/04847; G06V40/28; G06F3/017; G06F3/0482
Verdict: High Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

An authoring tool that captures raw sensor data during a performers gesture to define reusable gesture controls for animating a walk-around costumed character.

Abstract

A method for facilitating and enhancing computer-based authoring of gesture definitions that are useful in controlling a walk-around character and other systems using gesture controls. The method includes, during performance of a gesture by a performer, collecting sets of raw sensor data each corresponding to differing parameters of the performance of the gesture. The method includes displaying a graphical user interface with a graphical plot of each of the sets of raw sensor data. The method includes receiving user input identifying which of the parameters to include in a gesture definition. The method includes, for the graphical plots associated with the chosen parameters receiving user input defining a starting position, an ending position, a maximum value, and a minimum value. The gesture may involve movement of a performer's arms, legs, hands, head, eyes, and so on in a particular manner.

Background

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block or schematic drawing of walk-around character costume with acontrol system or assembly adapted for use with the walk-around character costume (e.g., with an articulated or non-articulated head) to facilitate control or navigation of audio playback and/or animation of costume components such as lip and eye movements via onboard motors;

FIG. 2 is a functional block diagram of a gesture training or definition system during its use by an aesthetic director (e.g., a human user, operator, or “definer”) to create a computer model or definition of a gesture;

FIG. 3 is a flow chart showingsteps of a gesture training or definition process that may be completed by operation of the system of FIG. 2;

FIG. 4 is a simplified view of a GUI or portion of a GUI that may be provided to a definer during use of the system of FIG. 2 to assist in refining a definition of a gesture;

FIG. 5 provides graphical representations of a hand as it moves through roll, pitch, and yaw and with acceleration relative to the X, Y, and Z axes;

FIG. 6 illustrates a monitor/display element operating to display a window/GUI generated by the authoring tool of the present description;

FIG. 7 illustrates the monitor/display element operating to display an analyzer window/GUI generated by the authoring tool;

FIG. 8 illustrates an authoring tool window or GUI when the authoring tool is in the edit gestures oper

Claims

1. A method for authoring gesture definitions for use in gesture-based control systems, comprising: during performance of a gesture by a performer, collecting a plurality of sets of raw sensor data each corresponding to differing parameters of theperformance of the gesture; displaying a graphical user interface (GUI) on a display device, wherein the GUI includes a graphical plot of each of the sets of raw sensor data; receiving and storing in memory user input identifying a subset of the parameters to include in a gesture definition; and for each of the graphical plots for the subset of the parameters in the gesture definition, receiving and storing, in the memory in the gesture definition, user input defining at least one of a starting position, an ending position, a maximumvalue, and a minimum value. 11. A system for authoring gestures for use by a gesture-based controller, comprising: a computer with a processor managing a display and a memory device; a gesture monitoring assembly including a set of sensors sampling data during performance of a gesture by a performer; and a gesture authoring tool run by the processor to: during the performance of the gesture by the performer, collect sets of sensor data from eachof the sensors; display a graphical user interface (GUI) on a display device, wherein theGUI includes a graphical plot of each of the sets of sensor data; receiving and storing in the memory device user input identifying a subset of the sensors to include in a gesturedefinition; and for each of the graphical plots for the subset of the sensors included inthe gesture definition, receiving and storing, in the memory device as part of the gesture definition, user input defining at least one of a starting position, an ending position,a maximum value, and a minimum value. 16. A gesture authoring method, comprising: collecting a set of data from a sensor during a data collection window; receiving first user input defining a gesture performance time period with a start time and an end time for performance of gesture by a performer during the collecting; creating a first gesture definition including the set of data from the sensor during the gesture performance time period; receiving second user input marking in the first gesture definition at least one of a start position, an end position, a maximum value, and a minimum value; and creating a second gesture definition comprising a modified version of the first gesture definition including the marking from the second user input.