- Number
- 9778741
- Published
- 2017-10-03
- Filed
- 2015-05-13
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Smoot; Lanny S., Smithwick; Quinn Y., Reetz; Daniel, Ilardi; Michael J.
- CPC
- G06T19/20; H04N13/156; G06T19/006; G06F3/017; G06F3/011; H04N13/363; H04N13/366
- Verdict
- High Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
A 3D immersive artistic experience that captures a participants 3D image, keys them out of the background, and composites them into a 3D virtual scene in real time.
Abstract
A method for providing a three dimensional (3D) drawing experience. The method includes capturing a 3D image of a participant and then processing this image to key the participant's image from a background. The keyed participant's image is mixed with a 3D background image such as frames or scenes from a 3D movie, and the mixed 3D image is projected on a projectionscreen. For example, left and right eye images may be projected from a pair of projectorswith polarization films over the lenses, and the projection screen may be a polarization-maintaining surface such as a silver screen. The user moves a drawing instrument in space in front of the projection screen, and spatial tracking performed to generate a locus of 3D positions. These 3D positions are used to create a 3D drawing image that is projected with the 3D background and participant images in real time.
Background
BRIEF DESCRIPTION OF THE DRAWINGS(1) FIG. 1 is a functional block drawing of a 3D drawing system according to one embodiment of the invention illustrating logically components of the system and their functionality to provide an immersive 3D drawing environment;(2) FIGS. 2-5 illustrate perspective views of a 3D drawing system during use during the capture of a 3D image of a user/artist, after the captured 3D user image has been inserted into a 3D background image on a 3D projection screen, during 3D drawing by the user/artist moving a wand or drawing instrument in 3D drawing space in front of the projection screen, and during a special effect involving removal of the previously inserted 3D user image and then spinning out of the completed 3D drawing image from the 3D background image; and(3) FIG. 6 is a flow diagram of a 3D drawing method such as may be implementedby operation of the systems of FIGS. 1-5.DETAILED DESCRIPTION(4) The following description is directed to systems and methods for providing users or “artists” with an immersive 3Ddrawing experience. The description begins with a discussion of a functional block diagram of an exemplary 3D drawing system with reference to FIG. 1 and then proceeds to a discussion of perspective views of one implemented 3D drawing system during use by an artist or user.(5) As will become clear from this description, a 3D drawing system is described thatfunctions to capture still or video 3D images of an artist and then projects these
Claims
1. A three dimensional (3D) drawing method for immersing a user in an interactive 3D experience, comprising: inresponse to first signals from a controller, projecting a stereoscopic 3D image of the user on a projection screen; with a position tracking mechanism, determining a set of 3D positions of a drawing instrument positionable by the user, during a drawing time period, relative to the 3D image and proximate but spaced apart from the projection screen; storing the set of determined 3D positions in memory accessible by a controller; with the controller, generating a 3D drawing image based on the set of determined 3D positions; and in response to second signals from the controller, projecting a mixed stereoscopic 3D image, on the projection screen, that comprises a blending of the projected stereoscopic 3D image of the user and the 3D drawing image, wherein the projected stereoscopic 3D image of the user comprises a first set of left and right eye images, wherein the projected mixed stereoscopic 3D image comprises a second set of left and right eye images differing from the first set of left and right images, wherein the projecting of the mixed stereoscopic 3D image is performed concurrently with the tracking of the 3D positions of the drawing instrument, and wherein the 3D drawing image is processed to include left and right images mapped to theright and left eye images of the 3D image of the user, whereby portions of the 3D drawingimage behind the projected stereoscopic 3D image of the user are occluded from view by the user.
7. A 3D drawing method, comprising: projecting a 3D background image, comprising left and right eye images providing a first stereo effect, on a projection screen; trackingspatial movement of a portion of drawing element moved by a user located proximate to theprojection screen, the tracking including determining a locus of 3D positions of the drawing element portion relative to the projection screen; generating a 3D drawing image from the locus of 3D positions; mixing the 3D drawing image with the 3D background image to create a mixed 3D image; and at least partially concurrently with the tracking of the spatialmovement of the drawing element portion, projecting the mixed 3D image, comprising left and right images providing a second stereo effect, on the projection screen.
13. A 3D drawing system for providing a participant with an immersive experience with real time feedback, comprising: an image capture assembly operating to capture 3D images of the participant;a controller accessing memory to retrieve 3D background images; a video mixer operated bythe controller to mix the 3D participant images and the 3D background images to form mixed 3D images; a projection screen; a 3D projection assembly projecting the mixed 3D images,including left and right eye images providing a first stereo effect, on the projection screen; a spatial tracking mechanism determining a locus of 3D positions relative to the projection screen; and a 3D drawing module operated by the controller to generate a 3D drawing image from the locus of 3D positions, wherein the controller further operates the video mixer to modify the mixed 3D images to include the 3D drawing image to provide a second stereo effect with the projection assembly including the 3D drawing image being concurrentlydisplayed with the 3D images of the participant in a 3D space in front of the projection screen and wherein the 3D drawing image includes left and right images mapped to the 3D image of the user, whereby portions of the 3D drawing image behind the 3D image of the user in the mixed 3D images projected by the 3D projection assembly are occluded from view by the user.