- Number
- 11600047
- Published
- 2023-03-07
- Filed
- 2018-07-17
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Guay; Martin et al.
- CPC
- G06T19/006; G06T7/73
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
AR image augmentation using a virtual character technique.
Abstract
An image processing system includes a computing platform having a hardware processor and a system memory storing an image augmentation software code, a three-dimensional (3D) shapes library, and/or a 3D poses library. The image processing system also includes a two-dimensional (2D) pose estimation module communicatively coupled to the image augmentation software code. The hardware processor executes the image augmentation software code to provide an image to the 2D pose estimation module and to receive a 2D pose data generated by the 2D pose estimation module based on the image. The image augmentation software code identifies a 3D shape and/or a 3D pose corresponding to the image using an optimization algorithm applied to the 2D pose data and one or both of the 3D poses library and the 3D shapes library, and may output the 3D shape and/or 3D pose to render an augmented image on a display.
Background
BACKGROUND (1) Augmented reality (AR), in which real world objects and/or environments are digitally augmented with virtual imagery, offers more immersive and enjoyable educational or entertainment experiences. SUMMARY (2) There are provided systems and methods for performing automated image augmentation using a virtual character, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.
Claims
1. An image processing system comprising: a computing platform including a hardware processor and a system memory; the system memory storing an image augmentation software code, a three-dimensional (3D) poses library including a plurality of poses, and a 3D body shapes library including a plurality of body shapes; a two-dimensional (2D) pose estimation software having computing instructions; the hardware processor configured to execute the image augmentation software code to: obtain an image including a real-world environment image portion corresponding to a real-world environment, a marker image portion corresponding to a marker in the real-world environment, and a human image portion corresponding to a human in the real-world environment; provide at least one portion of the image including the human image portion as an input to the 2D pose estimation software; receive from the 2D pose estimation software, a 2D pose data generated based on the at least one portion of the image including the human image portion; identify a 3D pose and a 3D body shape corresponding to the at least one portion of the image including the human image portion, using the 2D pose data, the plurality of poses in the 3D poses library and the plurality of body shapes in the 3D body shapes library, wherein the 3D pose is initialized to fit inside a bounding box of the marker; and generate a virtual character having the identified 3D pose and the identified 3D body shape. ||
11. A method for use by an image processing system including a computing platform having a hardware processor and a system memory storing an image augmentation software code, a three-dimensional (3D) poses library including a plurality of poses, and a 3D body shapes library including a plurality of body shapes, the method comprising: obtaining, by the image augmentation software code executed by the hardware processor, an image including a real-world environment image portion corresponding to a real-world environment, a marker image portion corresponding to a marker in the real-world environment, and a human image portion corresponding to a human in the real-world environment; providing, by the image augmentation software code executed by the hardware processor, at least one portion of the image including the human image portion as an input to the 2D pose estimation software; receiving from the 2D pose estimation software, by the image augmentation software code executed by the hardware processor, a 2D pose data generated based on the at least one portion of the image including the human image portion; identifying, by the image augmentation software code executed by the hardware processor, a 3D pose and a 3D body shape corresponding to the at least one portion of the image including the human image portion, using the 2D pose data, the plurality of poses in the 3D poses library and the plurality of body shapes in the 3D body shapes library, wherein the 3D pose is initialized to fit inside a bounding box of the marker; and generating, by the image augmentation software code executed by the hardware processor, a virtual character having the identified 3D pose and the identified 3D body shape.