Detection of contacts among event participants

Number: 11514677
Published: 2022-11-29
Filed: 2020-10-29
Assignee: Disney Enterprises, Inc.
Inventors: Kennedy; Justin Ali, Prince; Kevin John, Dietrich; Carlos Augusto, Van Dall; Dirk Edward
CPC: G06N3/09; G06N3/02; G06V20/41; G06T7/70; G06N3/0464; G06T7/75; G06N3/08; G06V40/23; G06T7/251; G06V10/82; G06N3/045; G06V20/53
Verdict: Set aside contact-tracing/proximity analytics, business/safety ops
Source: Google Patents · FreePatentsOnline

Abstract

Systems and methods are presented for detecting physical contacts effectuated by actions performed by an entity participating in an event. An action, performed by the entity, is detected based on a sequence of pose data associated with the entity's performance in the event. A contact with another entity in the event is detected based on data associated with the detected action. The action and the contact detections are employed by neural-network based detectors.

Background

BACKGROUND (1) In the computer vision field, human action recognition has been attempted using three-dimensional skeleton data, but many challenges remain in developing practical systems that are able to reliably perform such action recognition. Real-time automatic detection of actions from video is a complex problem, both in terms of accuracy and speed. For example, existing methods in computer vision may be capable of addressing the problem of classification, but are ill-suited for the problem of action detection in real-time settings, such as detecting actions during live events or making other real-time observations from live video feeds. (2) Detection of contacts effectuated by an entity participating (a participant) in an event captured by a live video is one example of a current human action recognition problem. This problem is complicated by the high motion in which the participant moves and by self-occlusion or occlusions by other participants. Current systems and methods may not be capable of real time analyses of spatiotemporal video and extraction of three-dimensional data therefrom to facilitate real time detection of contacts and other action recognitions among participants during live events.

Claims

1. A method comprising: receiving a sequence of pose data associated with a performance of an entity in an event; extracting a segment of the pose data from the sequence of the pose data; generating one or more feature maps derived from the segment of the pose data; detecting, by employing a neural-network-based action detector using the one or more feature maps, an action performed by the entity; and detecting, by employing a neural-network-based contact detector using the one or more feature maps, a contact between the entity and another entity in the event. || 11. An apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to: receive a sequence of pose data associated with a performance of an entity in an event; extract a segment of the pose data from the sequence of the pose data; generate one or more feature maps derived from the segment of the pose data; detect, by employing a neural-network-based action detector using the one or more feature maps, an action performed by the entity; and detect, by employing a neural-network-based contact detector using the one or more feature maps, a contact between the entity and another entity in the event. || 17. A non-transitory computer-readable medium comprising instructions executable by a processor to perform a method, the method comprising: receiving a sequence of pose data associated with a performance of an entity in an event; extracting a segment of the pose data from the sequence of the pose data; generating one or more feature maps derived from the segment of the pose data; detecting, by employing a neural-network-based action detector using the one or more feature maps, an action performed by the entity; and detecting, by employing a neural-network-based contact detector using the one or more feature maps, a contact between the entity and another entity in the event.