Real time surface augmentation using projected light

Number: 9940753
Published: 2018-04-10
Filed: 2016-10-11
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Grundhöfer; Anselm; Bermano; Amit
CPC: G06T11/00; G06T19/006; G06T7/33; G06T7/50; G06T7/73; G06V40/174; H04N23/20
Verdict: High Hardware
Source: Google Patents · FreePatentsOnline

The keeper's note

Real-time surface projection augmentation.

Abstract

A method of augmenting a target object with projected light is disclosed. The method includes determining a blend of component attributes to define visual characteristics of the target object, modifying an input image based, at least in part, on an image of the target object, wherein the modified input image defines an augmented visual characteristic of the target object, determining a present location of one or more landmarks on the target object based, at least in part, on the image of the target object, predicting a future location of the one or more landmarks, deforming a model of the target object based on the future location of the one or more landmarks, generating an augmentation image based on the deformed model and the modified input image, and transmitting for projection the augmentation image.

Background

BACKGROUND(1) Manipulating the appearance of surfaces with high frequency motion or rapid deformation is desirable in many fields such as the visual and performing arts. For example, manipulating the appearance of human faces is a common practice in many cultures and industries. Traditionally, appearances are altered through the application of paint, die, makeup, prosthetics, etc. Such alterations are static, and manual labor is typically required to apply and change the alteration. Projected light has also been used to alter the appearance of static, stationary surfaces, such as movie screens. However, realistic real-time projection onto moving or changing surfaces has proved difficult. Slow processing speeds and inefficient methods of calculation and projection have historically prevented realistic real-time projection onto moving or changing surfaces. To allow performers to move at the their natural pace, more efficient methods of computing projectable images to augment a moving surface in real time without perceptible lag are needed.SUMMARY(2) An example of the present disclosure relates to a method of augmenting a target object with projected light. The method includes determining, by a processor, a blend of component attributes to define visual characteristics of the target object, modifying, by the processor, an input image based, at least in part, on an image of the target object, wherein the modified input image defines an augmented visual characteristic of the targe

Claims

1. A method of augmenting a target object with projected light comprising: determining, by a processor, a blend of component attributes to define visual characteristics of the target object; modifying, by the processor, an input image based, at least in part, on an image of the target object, wherein the modified input image defines an augmented visual characteristic of the target object; determining, by the processor, a present location of one or more landmarks on the target object based, at least in part, on the image of the target object; predicting a future location of the one or more landmarks; deforming, by the processor, a model of the target object based on the future location of the one or more landmarks; generating, by the processor, an augmentation image based on the deformed model and the modified input image; and transmitting for projection, by the processor, the augmentation image. 8. A visual augmentation system comprising: a camera; a projector; and a processor coupled to the camera and the projector, the processor configured to: receive, from the camera, an image of a target object having a visual appearance; determine a weighted blend of component attributes to approximate the appearance target object based on the received image; modify a texture based on the weighted blend of component attributes; detect a location of at least one landmark in the image of the target object; predict a future location of the at least one landmark; generate a deformed model of the target object based, at least in part, on the future location of the at least one landmark; apply the texture to the deformed model to generate a projectable image; and transmit for projection by the projector, the projectable image. 17. A method of augmenting a face with projected light comprising: receiving, by a processor, an image of a face, wherein the image is of infrared light reflected from the face; analyzing the image of the face, by the processor, to independently detect current locations of landmarks and a facial expression; altering, by the processor, an appearance of an input image based, on the facial expression; predicting, by the processor, future locations of the landmarks based on the current locations of the landmarks; deforming, by the processor, a digital representation of the face based on the locations of the landmarks; applying, by the processor, the altered input image to the deformed digital representation of the face to generate an augmentation image; and transmitting, by the processor, the augmentation image for display.