Detection of contacts among event participants

Number: 11790652
Published: 2023-10-17
Filed: 2022-10-24
Assignee: Disney Enterprises, Inc.
Inventors: Kennedy; Justin Ali et al.
CPC: G06N3/09; G06T7/70; G06V10/82; G06V20/41; G06N3/0464; G06V40/23; G06N3/02; G06V20/53; G06N3/045; G06N3/08; G06T7/75; G06T7/251
Verdict: Set aside contact-tracing/proximity analytics, business/safety ops
Source: Google Patents · FreePatentsOnline

Abstract

Systems and methods are presented for detecting physical contacts effectuated by actions performed by an entity participating in an event. An action, performed by the entity, is detected based on a sequence of pose data associated with the entity's performance in the event. A contact with another entity in the event is detected based on data associated with the detected action. The action and the contact detections are employed by neural-network based detectors.

Background

BACKGROUND (1) In the computer vision field, human action recognition has been attempted using three-dimensional skeleton data, but many challenges remain in developing practical systems that are able to reliably perform such action recognition. Real-time automatic detection of actions from video is a complex problem, both in terms of accuracy and speed. For example, existing methods in computer vision may be capable of addressing the problem of classification, but are ill-suited for the problem of action detection in real-time settings, such as detecting actions during live events or making other real-time observations from live video feeds. (2) Detection of contacts effectuated by an entity participating (a participant) in an event captured by a live video is one example of a current human action recognition problem. This problem is complicated by the high motion in which the participant moves and by self-occlusion or occlusions by other participants. Current systems and methods may not be capable of real time analyses of spatiotemporal video and extraction of three-dimensional data therefrom to facilitate real time detection of contacts and other action recognitions among participants during live events.

Claims

1. A method comprising: receiving positional data associated with a performance of an entity; generating, using the positional data, one or more feature maps; training a neural-network-based action detector based on the one or more feature maps and respective action classes; and training a neural-network-based contact detector based on a modified version of the one or more feature maps and respective binary classes, wherein the modified version is based on the respective action classes. || 8. An apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to: receive positional data associated with a performance of an entity; generate, using the positional data, one or more feature maps; train a neural-network-based action detector based on the one or more feature maps and respective action classes; and train a neural-network-based contact detector based on a modified version of the one or more feature maps and respective binary classes, wherein the modified version is based on the respective action classes. || 15. A method comprising: receiving a sequence of positional data associated with a performance of an entity in an event; extracting a segment of the positional data from the sequence of the positional data; generating one or more feature maps derived from the segment of the positional data; detecting, by employing a neural-network-based action detector using the one or more feature maps, an action performed by the entity; and detecting, by employing a neural-network-based contact detector using the one or more feature maps, a contact between the entity and another entity in the event. || 21. An apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to: receive a sequence of positional data associated with a performance of an entity in an event; extract a segment of the positional data from the sequence of the positional data; generate one or more feature maps derived from the segment of the positional data; detect, by employing a neural-network-based action detector using the one or more feature maps, an action performed by the entity; and detect, by employing a neural-network-based contact detector using the one or more feature maps, a contact between the entity and another entity in the event.