- Number
- 10373379
- Published
- 2019-08-06
- Filed
- 2015-08-20
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Sumner; Robert, Ngo; Dat Tien, Nitti; Maurizio, Marra; Alessia, Magnenat; Stephane, Ryffel; Mattia
- CPC
- G06T7/11; G06T19/006; G06V20/647; G06T7/10; G06T7/149
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Deformable-surface tracking based AR image generation grant.
Abstract
There are provided systems and methods for performing deformable-surface tracking based augmented reality image generation. In one implementation, such a system includes a hardware processor and a system memory storing an augmented reality three-dimensional image generator. The hardware processor is configured to execute the augmented reality three-dimensional image generator to receive image data corresponding to a two-dimensional surface, and to identify an image template corresponding to the two-dimensional surface based on the image data. In addition, the hardware processor is configured to execute the augmented reality three-dimensional image generator to determine a surface deformation of the two-dimensional surface. The hardware processor is further configured to execute the augmented reality three-dimensional image generator to generate an augmented reality three-dimensional image including at least one feature of the two-dimensional surface, based on the image template and the surface deformation of the two-dimensional surface.
Background
BACKGROUND(1) The visual richness and immersiveness provided by modern information and entertainment technologies have increasingly led users to abandon more traditional, tangible media, especially print media such as books, in favor of electronic media content. Nevertheless, some forms of tangible media have resisted translation into electronic form due to the physically interactive way that users of such tangible media typically engage with them. For example, an artist's tactile experience while marking a canvas, or that of a child marking a coloring book may not be easily replicated through use of a conventional electronic user interface. Thus, a real-time solution enabling generation of augmented reality images from tangible images produced or modified by hand is desirable in order to more fully extend the visual richness and immersiveness enabled by use of electronic media to the creative activities of artists and children.SUMMARY(2) There are provided systems and methods for deformable-surface tracking based augmented reality image generation, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.
Claims
1. A system comprising: a hardware processor; a system memory having an augmented reality three-dimensional image generator stored therein; wherein the hardware processor is configured to execute the augmented reality three-dimensional image generator to: receive image data from a video frame, the image data corresponding to a two-dimensional surface; remove colors from the image data by transforming the image data from a red-green-blue (RGB) color space to a hue-saturation-lightness (HSV) color space to provide a gray scale image data; detect a plurality of feature points in the gray scale image data; extract a feature point descriptor for each detected one of the plurality of feature points; identify an image template corresponding to the two-dimensional surface based on highest votes from the extracted feature point descriptors; determine a surface deformation of the two-dimensional surface, wherein determining the surface deformation includes representing the two-dimensional surface by a two-dimensional mesh, regularizing the two-dimensional mesh, and performing an outlier rejection based on the regularized two-dimensional mesh; generate an augmented reality three-dimensional image including at least one feature of the two-dimensional surface based on the image template and the surface deformation of the two-dimensional surface; and predict, using a linear motion model, a three-dimensional pose of the two-dimensional surface in a next video frame, and create an occlusion mask for a surface projection in image data of the next video frame. |
7. A method for use by a system including a hardware processor and an augmented reality three-dimensional image generator stored in a system memory, the method comprising: receiving, using the hardware processor, image data corresponding to a two-dimensional surface; removing, using the hardware processor, colors from the image data by transforming the image data from a red-green-blue (RGB) color space to a hue-saturation-lightness (HSV) color space to provide a gray scale image data; detecting, using the hardware processor, a plurality of feature points in the gray scale image data; extracting, using the hardware processor, a feature point descriptor for each detected one of the plurality of feature points; identifying, using the hardware processor, an image template corresponding to the two-dimensional surface based on highest votes from the extracted feature point descriptors; determining, using the hardware processor, a surface deformation of the two dimensional surface, wherein the determining includes representing the two-dimensional surface by a two-dimensional mesh, regularizing the two-dimensional mesh, and performing an outlier rejection based on the regularized two-dimensional mesh; generating, using the hardware processor, an augmented reality three-dimensional image including at least one feature of the two-dimensional surface based on the image template and the surface deformation of the two-dimensional surface; and predicting, using a linear motion model, a three-dimensional pose of the two-dimensional surface in a next video frame, and create an occlusion mask for a surface projection in image data of the next video frame.