Hierarchical model for human activity recognition

Number: 9846845
Published: 2017-12-19
Filed: 2012-11-21
Assignee: Disney Enterprises, Inc.
Inventors: Sigal; Leonid, Lan; Tian
CPC: G06N20/00
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

A hierarchical machine-learning model that recognizes human activity at multiple levels of detail, trainable to a users preferred inference granularity.

Abstract

The disclosure provides an approach for recognizing and analyzing activities. In one embodiment, a learning application trains parameters of a hierarchical model which represents human (or object) activity at multiple levels of detail. Higher levels of detail may consider more context, and vice versa. Further, learning may be optimized for a user-preferred type of inference by adjusting a learning criterion. An inference application may use the trained model to answer queries about variable(s) at any levelof detail. In one embodiment, the inference application may determine scores for each possible value of the query variable by finding the best hierarchical event representation that maximizes a scoring function while fixing the value of the query variable to its possible values. Here, the inference application may approximately determine the best hierarchical event representation by iteratively optimizing one level-of-detail variable at a time while fixing other level-of-detail variables, until convergence.

Background

BRIEF DESCRIPTION OF THE DRAWINGS(1) So that the manner in which the above recited aspects are attained and can be understood in detail, a more particular description of aspects of the invention, briefly summarized above, may be had by reference to the appended drawings.(2) It is to be noted, however, that the appended drawings illustrate only typical aspects of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective aspects.(3) FIG. 1A illustrates an example hierarchical model for recognizing and analyzing activities, according to an embodiment of the invention.(4) FIG. 1B illustrates an example graph for representing mid-level social roles and interactions, according to an embodiment of the invention.(5) FIG. 2 illustrates a method for determining social roles from multi-person scenes using a hierarchical model, according toan embodiment of the invention.(6) FIG. 3 depicts a block diagram of a system in which anembodiment may be implemented.DETAILED DESCRIPTION(7) Embodiments disclosed herein provide techniques for recognizing and analyzing activities. Although, in describing this invention the focus is on activities related to team games, embodiments of this invention may beapplied to any activity or interaction involving a group of objects. In one embodiment, alearning application trains parameters of a hierarchical model, based on annotated example execution(s) (i.e., occurrences) of the game

Claims

1. A computer-implemented method for recognizing and analyzing activities, comprising: learning, via one or more processors, parameters of a classifier in a training operation based on feature vectors and activity elements corresponding to objects during one or more annotated example executions, wherein the classifier represents activity elements at multiple levels of detail; extracting feature vectors corresponding to one or more objects that interact during atest execution; and determining, based on the extracted feature vectors and using the classifier, activity elements associated with the one or more objects, at the multiple levelsof detail. 11. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for recognizing and analyzing activities, the operations comprising: learning, via one or more processors, parameters of a classifier in a training operation based on feature vectors and activity elements corresponding to objects during one or more annotated example executions, wherein the classifierrepresents activity elements at multiple levels of detail; extracting feature vectors corresponding to one or more objects that interact during a test execution; and determining, based on the extracted feature vectors and using the classifier, activity elements associated with the one or more objects, at the multiple levels of detail. 21. A system, comprising: a processor; and a memory, wherein the memory includes an application program configured to perform operations for recognizing and analyzing activities, the operations comprising: learning parameters of a classifier in a training operation based on feature vectors and activity elements corresponding to objects during one or more annotated example executions, wherein the classifier represents activity elements at multiple levels of detail, extracting feature vectors corresponding to one or more objects that interact during a test execution, and determining, based on the extracted feature vectors and using the classifier, activity elements associated with the one or more objects, at the multiple levels of detail. 22. A computer-implemented method for recognizing and analyzing activities in a set of video frames, comprising: learning parameters of a hierarchical model based on learning criteria, wherein the hierarchical model includes a plurality of levels of detail at whichactivity elements are modeled; identifying objects and extracting features corresponding to the identified objects; and determining, based on at least the extracted features and the hierarchical model, an answer to a query, wherein the query relates to one or more of the activity elements at one or more of the levels of detail.