- Number
- 10362299
- Published
- 2019-07-23
- Filed
- 2017-08-28
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Niemeyer; Gunter D., Smoot; Lanny S., Chawda; Vinay, Xi-Jie Pan; Matthew Keith
- CPC
- G06F3/011; B25J13/00; G06F3/017; H04N13/344; G06F3/012; G06F3/014; B25J13/089; G06F3/04815; B25J11/003
- Verdict
- High Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Physical experiences into VR worlds (robotic haptic interface, Niemeyer).
Abstract
A system for providing a user of a virtual reality (VR) system with physical interactions with an object in the real world or in the surrounding physical space while they are concurrently interacting in the virtual world with a corresponding virtual object. The real world object is dynamic with the system including a physical interaction system that includes a robot with a manipulator for moving, positioning, and/or orienting the real world object so as to move it into contact with the user. For example, the physical object is moved into contact with a tracked body part of the user, such as a hand, a tracked contact surface on the user's body, and so on, at a time that is accurately synchronized with a time of an interaction event occurring in the virtual world being created by the VR system.
Background
BACKGROUND1. Field of the Description(1) The present description relates, in general, to virtual reality (VR) systems and their uses to generate unique user experiences, and, more particularly, to systems and methods for providing the user of a VR system with physical experiences or haptic interactions while they are experiencing a VR world or environment provided by a VR system.2. Relevant Background(2) There is a growing interest in virtual reality (VR) games and applications in the entertainment and education industries. VR typically refers to computer technologies that use virtual reality headsets (or head mounted displays (HMDs)) to generate realistic images, sounds, and other sensations that replicate a real environment or create an imaginary setting. VR also simulates a user's physical presence in this environment. VR has been defined as a realistic and immersive simulation of a three-dimensional 360-degree environment, created using interactive software and hardware, and experienced or controlled by movement of the body or as an immersive, interactive experience generated by a computer.(3) A person using virtual reality equipment is able to “look around” the artificial world, and, with higher quality VR systems, move about in it and virtually interact with features or items depicted in the headset. A virtual world or environment (or its associated imagery) is displayed with a virtual reality headset. VR headsets may include head-mounted goggles with a screen in front
Claims
1. A system for providing a dynamic physical interaction to a user during a virtual reality (VR) experience, comprising: a VR system including a headset with a display screen and a VR rendering module generating a video output, wherein the display screen displays an image of a virtual world based on the video output; and a physical interaction system comprising a robotic mechanism with an object manipulator and a robot control module generating commands for operating the robotic mechanism to selectively position a physical object within a space, wherein a wearer of the headset is positioned in the space, wherein the image of the virtual world includes an image of a virtual object corresponding to the physical object, wherein the commands generated by the robot control module cause the robotic mechanism to move the physical object in the space based on a state of the virtual object in the virtual world, wherein the commands generated by the robot control module cause the object manipulator to move the physical object into contact with a surface on a body part of the wearer at a first time, wherein the image of the virtual world includes an image of a virtual body part corresponding to the body part of the wearer, wherein the image of the virtual object is shown in the image of the virtual world to move into contact with the image of the virtual body part at a second time matching the first time, wherein the state of the virtual object in the virtual world includes a velocity, and wherein the commands cause the object manipulator to move the physical object through the space at a velocity matching the velocity of the virtual object in the virtual world. |
6. A system for enhancing a VR experience, comprising: a VR rendering module generating a video of a virtual world; a headset, wearable by a participate in the VR experience, with a screen displaying the video of the virtual world; a tracking system for tracking velocity and location of at least one portion of a body of the participant in a space; a physical object in the space, wherein the VR rendering module operates to include an image of a virtual object in the virtual world based corresponding to the physical object and an image of a virtual body part corresponding to the at least one portion of the body of the participant based on the velocity and the location from the tracking system; and a robotic mechanism moving the physical object in the space to the location of the at least one portion of the body of the participant, wherein the VR rendering module concurrently generates the video of the virtual world with the virtual object contacting the virtual body part, and wherein the robotic mechanism manipulates an orientation of the physical object prior to positioning the physical object at the location of the at least one portion of the body of the participant, whereby the orientation matches an orientation of the virtual object in the virtual world during the contacting with the virtual body part. |
11. A system for providing a VR experience with haptics, comprising: a VR system displaying an image of a virtual world to a VR user including a virtual object at a first location; a tracking system tracking a location of a contact surface on a body of the VR user, wherein the VR system includes an image of a virtual contact surface at a second location in the virtual world corresponding to the location of the contact surface on the body tracked by the tracking system; and a physical interaction system including a robot controller and a robotic mechanism, wherein the robot controller generates commands to operate the robotic mechanism to move a physical object from a location spaced apart a distance from the location of the contact surface to a new location whereby the physical object abuts the contact surface on the body of the VR user, wherein the VR system concurrently modifies the image of the virtual world to show the virtual object travel from the first location to the second location in the virtual world, whereby arrival of the physical object at the new location and arrival of the virtual object at the second location are time synchronized, wherein the VR system shows the virtual object traveling at a first velocity from the first location to the second location, and wherein the robot controller generates the commands to cause the robotic mechanism to move the physical object at a second velocity matching the first velocity when the physical object is caused to abut the contact surface of the body of the VR user. |
17. A system for providing a dynamic physical interaction to a user during a virtual reality (VR) experience, comprising: a VR system including a headset with a display screen and a VR rendering module generating a video output, wherein the display screen displays an image of a virtual world based on the video output; and a physical interaction system comprising a robotic mechanism with an object manipulator and a robot control module generating commands for operating the robotic mechanism to selectively position a physical object within a space, wherein a wearer of the headset is positioned in the space, wherein the image of the virtual world includes an image of a virtual object corresponding to the physical object, wherein the commands generated by the robot control module cause the robotic mechanism to move the physical object in the space based on a state of the virtual object in the virtual world, wherein the commands generated by the robot control module cause the object manipulator to move the physical object into contact with a surface on a body part of the wearer at a first time, wherein the image of the virtual world includes an image of a virtual body part corresponding to the body part of the wearer, wherein the image of the virtual object is shown in the image of the virtual world to move into contact with the image of the virtual body part at a second time matching the first time, wherein the body part includes a hand of the wearer of the headset, and wherein the commands cause the object manipulator to halt travel at a preset distance past location a tracked location of the hand at about the first time. |
18. A system for enhancing a VR experience, comprising: a VR rendering module generating a video of a virtual world; a headset, wearable by a participate in the VR experience, with a screen displaying the video of the virtual world; a tracking system for tracking velocity and location of at least one portion of a body of the participant in a space; a physical object in the space, wherein the VR rendering module operates to include an image of a virtual object in the virtual world based corresponding to the physical object and an image of a virtual body part corresponding to the at least one portion of the body of the participant based on the velocity and the location from the tracking system; and a robotic mechanism moving the physical object in the space to the location of the at least one portion of the body of the participant, wherein the VR rendering module concurrently generates the video of the virtual world with the virtual object contacting the virtual body part, wherein the robotic mechanism includes a gripper for selectively gripping and releasing the physical object, wherein the gripper releases the physical object within a predefined time period after the physical object is moved into the location of the at least one body part of the participant, and wherein the robotic mechanism releases the physical object after sensing a force greater than a predefined impact force has been imparted upon the at least one body part of the participant.