- Number
- 10068547
- Published
- 2018-09-04
- Filed
- 2012-06-29
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Sobeski; David; Leake; Brian; Mitchell; Kenny
- CPC
- G09G5/026
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
AR surface painting creative tool.
Abstract
Techniques for simulating interactions using an augmented reality device are described. Embodiments receive a request to paint over portions of a visual scene. Here, the visual scene is captured using one or more camera devices of the augmented reality device and is presented on a display of the augmented reality device. A first object in the visual scene corresponding to the first location is identified. Additionally, embodiments paint at least a portion of the first object as specified by the received request, and render a series of frames depicting the first object, such that the painted at least a portion of the first object is shown as painted when viewed from different perspectives using the augmented reality device.
Background
BACKGROUND(1) Field of the Invention(2) The present invention generally relates to a human-computer interface and more specifically to techniques for painting surfaces on an augmented reality device.(3) Description of the Related Art(4) Computer graphics technology has come a long way since video games were first developed. Relatively inexpensive 3D graphics engines now provide nearly photo-realistic interactive game play on hand-held video game, home video game and personal computer hardware platforms costing only a few hundred dollars. These video game systems typically include a hand-held controller, game controller, or, in the case of a hand-held video game platform, an integrated controller. A user or player uses the controller to send commands or other instructions to the video game system to control a video game or other simulation being played. For example, the controller may be provided with a manipulator (e.g., a joystick) and buttons operated by the user.(5) Many hand-held gaming devices include some form of camera device which may be used to capture an image or a series of images of a physical, real-world scene. The captured images can then be displayed, for instance, on a display of the hand-held gaming device. Certain devices may be configured to insert virtual objects into the captured images before the images are displayed. Additionally, other devices or applications may enable users to draw or paint particular within a captured image of a physical scene. Howe
Claims
1. A computer-implemented method to maintain conformity of a virtual painting onto a physical object captured and portrayed via an augmented reality device, even as the augmented reality device physically moves relative to the physical object in a physical environment over time, the computer-implemented method comprising: presenting, via a display of the augmented reality device, the physical object within a visual scene captured in the physical environment using one or more camera devices of the augmented reality device, the physical object having an area of a first color; receiving a request to at least partially fill the area using virtual paint of a second color different from the first color; responsive to the request, identifying the physical object depicted by the visual scene, based on at least one of: (i) one or more edges determined for the physical object from analyzing the visual scene; and (ii) predefined geometrical data specifying geometrical characteristics of a specified type of the physical object; determining one or more illumination characteristics of the physical environment; generating, based on an initial image of the identified physical object in the visual scene and based further on the one or more illumination characteristics of the physical environment, an initial virtual painting projected onto the area of the physical object in the initial image and as viewed from a first physical position of the augmented reality device relative to the physical object in the physical environment; and responsive to the augmented reality device being moved from the first physical position to a second physical position relative to the physical object in the physical environment, the second physical position being distinct from the first physical position, generating, based on an updated image of the identified physical object in the visual scene and based further on the one or more illumination characteristics of the physical environment, an updated virtual painting projected onto the area of the physical object in the updated image and as viewed from the second physical position of the augmented reality device, wherein the updated virtual painting is visually distinct from the initial virtual painting in terms of at least one visual aspect selected from size, shape, orientation, pattern, and color; wherein the updated virtual painting is generated in order to maintain conformity of the virtual painting onto the physical object even as the augmented reality device physically moves relative to the physical object in the physical environment over time, wherein the initial virtual painting and the updated virtual painting are output.
14. An augmented reality device to maintain conformity of a virtual painting onto a physical object captured and portrayed via the augmented reality device, even as the augmented reality device physically moves relative to the physical object in a physical environment over time, the augmented reality device comprising: a processor; and a memory containing a program that, when executed by the processor, performs an operation comprising: presenting, via a display of the augmented reality device, the physical object within a visual scene captured in the physical environment using one or more camera devices of the augmented reality device, the physical object having an area of a first color; receiving a request to at least partially fill the area using virtual paint of a second color different from the first color; responsive to the request, identifying the physical object depicted by the visual scene, based on at least one of: (i) one or more edges determined for the physical object from analyzing the visual scene; and (ii) predefined geometrical data specifying geometrical characteristics of a specified type of the physical object; determining one or more illumination characteristics of the physical environment; generating, based on an initial image of the identified physical object in the visual scene and based further on the one or more illumination characteristics of the physical environment, an initial virtual painting projected onto the area of the first physical object in the initial image and as viewed from a first physical position of the augmented reality device relative to the physical object in the physical environment; responsive to the augmented reality device being moved from the first physical position to a second physical position relative to the physical object in the physical environment, the second physical position being distinct from the first physical position, generating, based on an updated image of the identified physical object in the visual scene and based further on the one or more illumination characteristics of the physical environment, an updated virtual painting projected onto the area of the physical object in the updated image and as viewed from the second physical position of the augmented reality device, wherein the updated virtual painting is visually distinct from the initial virtual painting in terms of at least one visual aspect selected from size, shape, orientation, pattern, and color; wherein the updated virtual painting is generated in order to maintain conformity of the virtual painting onto the physical object even as the augmented reality device physically moves relative to the physical object in the physical environment over time, wherein the initial virtual painting and the updated virtual painting are output.
21. A non-transitory computer-readable medium containing a program that, when executed, performs an to maintain conformity of a virtual painting onto a physical object captured and portrayed via an augmented reality device, even as the augmented reality device physically moves relative to the physical object in a physical environment over time, the operation comprising: presenting, via a display of the augmented reality device, the physical object within a visual scene captured in the physical environment using one or more camera devices of the augmented reality device, the physical object having an area of a first color; receiving a request to at least partially fill the area using virtual paint of a second color different from the first color; responsive to the request, identifying the physical object depicted by the visual scene, based on at least one of: (i) one or more edges determined for the physical object from analyzing the visual scene; and (ii) predefined geometrical data specifying geometrical characteristics of a predefined type of the physical object; determining one or more illumination characteristics of the physical environment; generating, based on an initial image of the identified physical object in the visual scene and based further on the one or more illumination characteristics of the physical environment, an initial virtual painting projected onto the area of the physical object in the initial image and as viewed from a first physical position of the augmented reality device; responsive to the augmented reality device being moved from the first physical position to a second physical position relative to the physical object in the physical environment, the second physical position being distinct from the first physical position, generating, based on an updated image of the identified physical object in the visual scene and based further on the one or more illumination characteristics of the physical environment, an updated virtual painting projected onto the area the physical object in the updated image and as viewed from the second physical position of the augmented reality device, wherein the updated virtual painting is visually distinct from the initial virtual painting in terms of at least one visual aspect selected from size, shape, orientation, pattern, and color; wherein the updated virtual painting is generated in order to maintain conformity of the virtual painting onto the physical object even as the augmented reality device physically moves relative to the physical object in the physical environment over time, wherein the initial virtual painting and the updated virtual painting are output.