- Number
- 11024098
- Published
- 2021-06-01
- Filed
- 2020-06-30
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Drake; Corey D., Mitchell; Kenneth J., Rodgers; Rachel E., Hager, IV; Joseph G., McIntosh; Kyna P., Pan; Ye
- CPC
- G06T19/006; G06F3/011; G06F3/014; G06F3/017; G06T19/20
- Verdict
- Low Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Physical object augmented with virtual/AR components.
Abstract
Systems and methods are presented for immersive and simultaneous animation in a mixed reality environment. Techniques disclosed represent a physical object, present at a scene, in a 3D space of a virtual environment associated with the scene. A virtual element is posed relative to the representation of the physical object in the virtual environment. The virtual element is displayed to users from a perspective of each user in the virtual environment. Responsive to an interaction of one user with the virtual element, an edit command is generated and the pose of the virtual element is adjusted in the virtual environment according to the edit command. The display of the virtual element to the users is then updated according to the adjusted pose. When simultaneous and conflicting edit commands are generated by collaborating users, policies to reconcile the conflicting edit commands are disclosed.
Background
BACKGROUND (1) The present disclosure relates to the production of entertainment shows involving the choreographing of characters including physical and computer-generated components. (2) Animating an articulated character—i.e., determining the three-dimensional (3D) pose of each joint of the character in each key-frame of an animation clip—is a complex and time-consuming process. Conventional character animation for films or game production is performed with software applications—such as Maya, 3DS Max, and Blender—via two degrees of freedom (2DoF) interfacing means and two-dimensional (2D) display feedback. However, the process of creating a character's expressive motion through posing is limited by the 2D medium. Interfacing devices that allow for posing of a character through 3D spatial inputs, such as a Leap Motion device or physical objects (i.e., puppets), introduced improvements, but are still limited in providing a relative scale and perspective when animating in coordination with or relative to other show elements in the shared space of the scene. (3) Recent advances in Virtual Reality (VR) technologies provide users with an immersive virtual environment through which they can interact with VR elements generated by the VR system. Consumer VR systems, typically, include a head-mounted display (HMD) and handheld controllers equipped with sensors, with which the 3D location and orientation of the HMD and controllers in the virtual environment are tracked. Thus, a user,
Claims
1. A method comprising: representing a physical object, present at a physical scene, using a three-dimensional (3D) model in a virtual environment associated with the physical scene; providing a pose for a virtual element in the virtual environment at a location relative to the 3D model of the physical object; displaying the virtual element to a first user from a perspective of the first user in the virtual environment; generating a first edit command derived from data received from a controller operated by the first user; adjusting the pose of the virtual element in the virtual environment according to the first edit command; updating the displaying of the virtual element to the first user according to the adjusted pose; representing another physical object, present at the physical scene, using another 3D model in the virtual environment; detecting a distance, in the virtual environment, between the another 3D model of the another physical object and the virtual element; and alerting when the detected distance is below a predetermined value. ||
5. A method comprising: representing a physical object, present at a physical scene, using a three-dimensional (3D) model in a virtual environment associated with the physical scene; providing a pose for a virtual element in the virtual environment at a location relative to the 3D model of the physical object; displaying the virtual element to a first user from a perspective of the first user in the virtual environment; generating a first edit command derived from data received from a controller operated by the first user; adjusting the pose of the virtual element in the virtual environment according to the first edit command; displaying the virtual element to a second user from a perspective of the second user in the virtual environment; generating a second edit command derived from second data received from a second controller operated by the second user; determining a conflict between the first edit command and the second edit command; reconciling, in response to determining the conflict, the first edit command and the second edit command into a third edit command; further adjusting the adjusted pose of the virtual element in the virtual environment according to the third edit command; and further updating the displaying of the virtual element to the first user and to the second user according to the further adjusted pose. ||
11. An apparatus comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the apparatus to: represent a physical object, present at a physical scene, using a three-dimensional (3D) model in a virtual environment associated with the physical scene; provide a pose for a virtual element in the virtual environment at a location relative to the 3D model of the physical object; display the virtual element to a first user from a perspective of the first user in the virtual environment; generate a first edit command derived from data received from a controller operated by the first user; adjust the pose of the virtual element in the virtual environment according to the first edit command; update the displaying of the virtual element to the first user according to the adjusted pose; represent another physical object, present at the physical scene, using another 3D model in the virtual environment; detect a distance, in the virtual environment, between the another 3D model of the another physical object and the virtual element; and alert when the detected distance is below a predetermined value. ||
15. An apparatus comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the apparatus to: represent a physical object, present at a physical scene, using a three-dimensional (3D) model in a virtual environment associated with the physical scene; provide a pose for a virtual element in the virtual environment at a location relative to the 3D model of the physical object; display the virtual element to a first user from a perspective of the first user in the virtual environment; generate a first edit command derived from data received from a controller operated by the first user; adjust the pose of the virtual element in the virtual environment according to the first edit command; update the displaying of the virtual element to the first user according to the adjusted pose; display the virtual element to a second user from a perspective of the second user in the virtual environment; generate a second edit command derived from second data received from a second controller operated by the second user; determine a conflict between the first edit command and the second edit command; reconcile, in response to determining the conflict, the first edit command and the second edit command into a third edit command; further adjust the adjusted pose of the virtual element in the virtual environment according to the third edit command; and further update the displaying of the virtual element to the first user and to the second user according to the further adjusted pose.