- Number
- 10726628
- Published
- 2020-07-28
- Filed
- 2018-09-17
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Sumner; Robert, Ngo; Dat Tien, Nitti; Maurizio, Marra; Alessia, Magnenat; Stephane, Ryffel; Mattia
- CPC
- G06T7/11; G06T7/149; G06T19/006; G06V20/647; G06T7/10
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
CV deformable-surface tracking for AR image generation.
Abstract
There are provided systems and methods for performing deformable-surface tracking based augmented reality image generation. In one implementation, such a system includes a hardware processor and a system memory storing an augmented reality three-dimensional image generator. The hardware processor is configured to execute the augmented reality three-dimensional image generator to receive image data corresponding to a two-dimensional surface, and to identify an image template corresponding to the two-dimensional surface based on the image data. In addition, the hardware processor is configured to execute the augmented reality three-dimensional image generator to determine a surface deformation of the two-dimensional surface. The hardware processor is further configured to execute the augmented reality three-dimensional image generator to generate an augmented reality three-dimensional image including at least one feature of the two-dimensional surface, based on the image template and the surface deformation of the two-dimensional surface.
Background
BACKGROUND(1) The visual richness and immersiveness provided by modern information and entertainment technologies have increasingly led users to abandon more traditional, tangible media, especially print media such as books, in favor of electronic media content. Nevertheless, some forms of tangible media have resisted translation into electronic form due to the physically interactive way that users of such tangible media typically engage with them. For example, an artist's tactile experience while marking a canvas, or that of a child marking a coloring book may not be easily replicated through use of a conventional electronic user interface. Thus, a real-time solution enabling generation of augmented reality images from tangible images produced or modified by hand is desirable in order to more fully extend the visual richness and immersiveness enabled by use of electronic media to the creative activities of artists and children.SUMMARY(2) There are provided systems and methods for deformable-surface tracking based augmented reality image generation, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.
Claims
1. A system comprising: a hardware processor; a system memory having an augmented reality three-dimensional image generator stored therein; wherein the hardware processor is configured to execute the augmented reality three-dimensional image generator to: receive image data corresponding to a two-dimensional surface; remove colors from the image data by transforming the image data from a red-green-blue (RGB) color space to a hue-saturation-lightness (HSV) color space to provide a luminance image data; identify an image template corresponding to the two-dimensional surface based on the luminance image data; determine a surface deformation of the two-dimensional surface, wherein determining the surface deformation includes representing the two-dimensional surface by a two-dimensional mesh, regularizing the two-dimensional mesh, and performing an outlier rejection based on the regularized two-dimensional mesh; after performing the outlier rejection, apply a three-dimensional optimization to the surface deformation of the two-dimensional surface; generate an augmented reality three-dimensional image using the three-dimensional optimized surface deformation of the two-dimensional surface and based on the image template; and reproduce one or more of the colors from the image data in the augmented reality three-dimensional image.
8. A method for use by a system including a hardware processor and an augmented reality three-dimensional image generator stored in a system memory, the method comprising: receiving, using the hardware processor, image data corresponding to a two-dimensional surface; removing, using the hardware processor, colors from the image data by transforming the image data from a red-green-blue (RGB) color space to a hue-saturation-lightness (HSV) color space to provide a luminance image data; identifying, using the hardware processor, an image template corresponding to the two-dimensional surface based on the luminance image data; determining, using the hardware processor, a surface deformation of the two-dimensional surface, wherein determining the surface deformation includes representing the two-dimensional surface by a two-dimensional mesh, regularizing the two-dimensional mesh, and performing an outlier rejection based on the regularized two-dimensional mesh; after performing the outlier rejection, applying, using the hardware processor, a three-dimensional optimization to the surface deformation of the two-dimensional surface; generating, using the hardware processor, an augmented reality three-dimensional image using the three-dimensional optimized surface deformation of the two-dimensional surface and based on the image template; and reproducing, using the hardware processor, one or more of the colors from the image data in the augmented reality three-dimensional image.
15. A non-transitory computer-readable medium having stored thereon instructions, which when executed by a hardware processor, instantiate a method comprising: receiving image data corresponding to a two-dimensional surface; removing colors from the image data by transforming the image data from a red-green-blue (RGB) color space to a hue-saturation-lightness (HSV) color space to provide a luminance image data; identifying an image template corresponding to the two-dimensional surface based on the luminance image data; determining a surface deformation of the two-dimensional surface, wherein determining the surface deformation includes representing the two-dimensional surface by a two-dimensional mesh, regularizing the two-dimensional mesh, and performing an outlier rejection based on the regularized two-dimensional mesh; after performing the outlier rejection, applying a three-dimensional optimization to the surface deformation of the two-dimensional surface; generating an augmented reality three-dimensional image using the three-dimensional optimized surface deformation of the two-dimensional surface and based on the image template; and reproducing one or more of the colors from the image data in the augmented reality three-dimensional image.