Augmenting a physical object with virtual components

Number: 11475647
Published: 2022-10-18
Filed: 2021-04-28
Assignee: Disney Enterprises, Inc.
Inventors: Drake; Corey D., Mitchell; Kenneth J., Rodgers; Rachel E., Hager, IV; Joseph G., McIntosh; Kyna P., Pan; Ye
CPC: G06T19/20; G06F3/014; G06T19/006; G06F3/011; G06F3/017
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

AR technique augmenting physical objects with virtual components (granted).

Abstract

Systems and methods are presented for immersive and simultaneous animation in a mixed reality environment. Techniques disclosed represent a physical object, present at a scene, in a 3D space of a virtual environment associated with the scene. A virtual element is posed relative to the representation of the physical object in the virtual environment. The virtual element is displayed to users from a perspective of each user in the virtual environment. Responsive to an interaction of one user with the virtual element, an edit command is generated and the pose of the virtual element is adjusted in the virtual environment according to the edit command. The display of the virtual element to the users is then updated according to the adjusted pose. When simultaneous and conflicting edit commands are generated by collaborating users, policies to reconcile the conflicting edit commands are disclosed.

Background

BACKGROUND (1) The present disclosure relates to the production of entertainment shows involving the choreographing of characters including physical and computer-generated components. (2) Animating an articulated character—i.e., determining the three-dimensional (3D) pose of each joint of the character in each key-frame of an animation clip—is a complex and time-consuming process. Conventional character animation for films or game production is performed with software applications—such as Maya, 3DS Max, and Blender—via two degrees of freedom (2DoF) interfacing means and two-dimensional (2D) display feedback. However, the process of creating a character's expressive motion through posing is limited by the 2D medium. Interfacing devices that allow for posing of a character through 3D spatial inputs, such as a Leap Motion device or physical objects (i.e., puppets), introduced improvements, but are still limited in providing a relative scale and perspective when animating in coordination with or relative to other show elements in the shared space of the scene. (3) Recent advances in Virtual Reality (VR) technologies provide users with an immersive virtual environment through which they can interact with VR elements generated by the VR system. Consumer VR systems, typically, include a head-mounted display (HMD) and handheld controllers equipped with sensors, with which the 3D location and orientation of the HMD and controllers in the virtual environment are tracked. Thus, a user,

Claims

1. A method comprising: representing a physical object, present at a physical scene, using a three-dimensional (3D) model in a virtual environment associated with the physical scene; providing a pose for a virtual element in the virtual environment at a location relative to the 3D model of the physical object; obtaining a first edit command and a second edit command; determining a conflict between the first edit command and the second edit command; reconciling, in response to determining the conflict, the first edit command and the second edit command to generate a third edit command; adjusting the pose of the virtual element in the virtual environment according to the third edit command; and displaying the virtual element in the virtual environment according to the adjusted pose. || 11. An apparatus comprising: a processor; and a memory storing instructions that, when executed by the at least one processor, cause the apparatus to: represent a physical object, present at a physical scene, using a three-dimensional (3D) model in a virtual environment associated with the physical scene; provide a pose for a virtual element in the virtual environment at a location relative to the 3D model of the physical object; obtain a first edit command and a second edit command; determine a conflict between the first edit command and the second edit command; reconcile, in response to determining the conflict, the first edit command and the second edit command to generate a third edit command; adjust the pose of the virtual element in the virtual environment according to the third edit command; and display the virtual element in the virtual environment according to the adjusted pose.