- Number
- 10359906
- Published
- 2019-07-23
- Filed
- 2017-07-25
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Sumner; Robert, Noris; Gioacchino
- CPC
- G06F3/0425; G06F3/04815; G06F3/0346
- Verdict
- Medium Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Haptic interface for population of 3D virtual environment.
Abstract
The disclosure provides an approach for populating a virtual environment with objects. In one embodiment, an editing application may track a handheld device using sensor data from a camera, by following an image displayed on the handheld device's screen. The editing application then updates the position of an object in the virtual environment according to the tracked position of the handheld device. Initially, the handheld device may be placed at a fixed location for calibration purposes, during which the editing application initializes a mapping between the virtual and physical environments. To add an object to the virtual environment, a user may select the object on the handheld device. The user may then place the object at a desired location and orientation in the virtual environment by moving the handheld device in the physical environment.
Background
BACKGROUNDField(1) This disclosure provides techniques for interacting with virtual environments. More specifically, embodiments of this disclosure present a haptic interface for populating three-dimensional virtual environments.Description of the Related Art(2) Experts typically manipulate objects in three-dimensional (3D) virtual environments using mouse gestures and keyboard commands, while working in different editing modes such as translation, rotation, scaling, and the like. For example, to move an object in the commercially available Maya® program, a user may move the view of a particular camera (orthographic or perspective), zoom in on the object, and translate the object within the camera view. While adequate for professionals, this form of 3D object manipulation is often difficult to learn and unsuitable for casual users and children, particularly as users are required to understand a non-trivial mapping between the space where the physical device is manipulated and the space of the virtual environment.(3) Simplified solutions have been proposed to facilitate 3D object manipulation by casual users and children. By limiting the user's choices, such as by forcing predefined positions or orientations, it is possible to simplify the 3D object manipulation process. However, such approaches can limit the expressiveness and depth of what is produced.SUMMARY(4) One embodiment of this disclosure provides a computer implemented method for populating a virtual environment. The
Claims
1. A computer-implemented method of using a handheld device in a physical environment to manipulate an object in a virtual environment, the computer-implemented method comprising: receiving one or more image frames captured by a camera; tracking movement of the handheld device in the physical environment, by operation of one or more computer processors and by determining a location and an orientation of a predefined image displayed on the handheld device, wherein the location and orientation are determined in each of the received image frames and based on one or more computer vision algorithms, wherein the handheld device is switched from not displaying the predefined image, to displaying the predefined image, when movement of the handheld device is to be tracked; and updating a location and orientation of the object in the virtual environment based on the tracked movement of the handheld device and based further on a mapping between the handheld device in the physical environment and the object in the virtual environment, whereafter an indication of the object having the updated location and orientation is output. |
12. A non-transitory computer-readable medium storing a program, executable to perform operations for using a handheld device in a physical environment to manipulate an object in a virtual environment, the operations comprising: receiving one or more image frames captured by a camera; tracking movement of the handheld device in the physical environment, by operation of one or more computer processors and by determining a location and an orientation of a predefined image displayed on the handheld device, wherein the location and orientation are determined in each of the received image frames and based on one or more computer vision algorithms, wherein the handheld device is switched from not displaying the predefined image, to displaying the predefined image, when movement of the handheld device is to be tracked; and updating a location and orientation of the object in the virtual environment based on at least the tracked movement of the handheld device and based further on a mapping between the handheld device in the physical environment and the object in the virtual environment, whereafter an indication of the object having the updated location and orientation is output. |
19. A system of using a handheld device in a physical environment to manipulate an object in a virtual environment, the system comprising: one or more computer processors; and a memory, wherein the memory includes an application program executable by the one or more computer processors to perform operations comprising: receiving one or more image frames captured by a camera, tracking movement of the handheld device in the physical environment by determining a location and an orientation of a predefined image displayed on the handheld device, wherein the location and orientation are determined in each of the received image frames and based on one or more computer vision algorithms, wherein the handheld device is switched from not displaying the predefined image, to displaying the predefined image, when movement of the handheld device is to be tracked, and updating a location and orientation of the object in the virtual environment based on the tracked movement of the handheld device and based further on a mapping between the handheld device in the physical environment and the object in the virtual environment, whereafter an indication of the object having the updated location and orientation is output.