- Number
- 10095953
- Published
- 2018-10-09
- Filed
- 2014-05-28
- Assignee
- Disney Enterprises, Inc.; ETH Zurich (Eidgenoessische Technische Hochschule Zurich)
- Inventors
- Smolic; Aljosa Aleksej Andrej; Aydin; Tunc Ozan; Poulakos; Steven Charles; Heinzle; Simon; Chapiro; Alexandre
- CPC
- H04N13/128; H04N13/111
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Depth-based 3D display processing.
Abstract
Techniques of depth modification for display applications are disclosed. A multiscopic video frame, including at least first and second images depicting a scene, is received. Depth information pertaining to the multiscopic video frame is received. A depth mapping function is determined based on one or more image characteristics of the multiscopic video frame, other than depth. The depth information is modified based on the depth mapping function. At least a third image is generated based on the modified depth information and based further on at least one of the first and second images, where the third image is output for display.
Background
CROSS-REFERENCE TO RELATED APPLICATIONS(1) This application is related to U.S. patent application Ser. No. 12/766,734 (now U.S. Pat. No. 8,711,204), filed Apr. 23, 2010, which claims benefit of U.S. provisional patent application Ser. No. 61/296,425, filed on Jan. 19, 2010, and U.S. provisional patent application Ser. No. 61/260,274, filed on Nov. 11, 2009. This application is also related to co-pending U.S. patent application Ser. No. 13/601,363, filed on Aug. 31, 2012, and which is a continuation-in-part of the above-identified U.S. patent application. Each of the aforementioned patent applications is herein incorporated by reference in its entirety.BACKGROUND(2) Human beings typically see using stereoscopic vision. The left and right eyes of an observer each perceives slightly different views of a scene, and the brain of the observer fuses the slightly different views into a single image that provides depth information. The depth information allows the observer to perceive the relative distance to various objects in the scene. Movies filmed with a single camera may not provide such depth information to the viewer and thus tend to look flat.(3) Early efforts in 3-D movie technology used anaglyphs, in which two images of the same scene, with a relative offset between them, are superimposed on a single piece of movie film, with the images being subject to complimentary color filters (e.g., red and green). Viewers donned special glasses so that one image would be seen only by
Claims
1. A computer-implemented method to programmatically conform multiscopic video content to restricted depth ranges supported by autostereoscopic displays, the computer-implemented method comprising: receiving a multiscopic video frame comprising at least first and second images depicting a scene; receiving depth information pertaining to the multiscopic video frame and exceeding a depth range supported by a target autostereoscopic display as measured based on a predefined perception criterion, wherein the supported depth range is smaller in extent than that of a non-autostereoscopic display, wherein the depth information comprises pixel-specific depth information represented as one or more grayscale images; determining a depth mapping function based on one or more image characteristics of the multiscopic video frame, other than depth, the one or more image characteristics including at least saliency and edges; modifying the depth information based on the depth mapping function and using: a mapping algorithm that at least partially preserves perceptible depth in the scene with a predefined degree of depth compression and by accounting for the depth range supported by the target autostereoscopic display, wherein the modified depth information conforms to the depth range supported by the target autostereoscopic display; at least partially preserving, in a third image, the saliency and edges of the multiscopic video frame, by generating the third image by operation of one or more computer processors and based on (i) the modified depth information and (ii) at least one of the first and second images; and extending a perceptible depth range of the scene by forming multiview output for the target autostereoscopic display, including augmenting the multiscopic video frame to include the generated third image, wherein the augmented multiscopic video frame is output for display via the target auto stereoscopic display.
7. A non-transitory computer-readable medium containing a program which, when executed, performs an operation to programmatically conform multiscopic video content to restricted depth ranges supported by autostereoscopic displays, the operation comprising: receiving a multiscopic video frame comprising at least first and second images depicting a scene; receiving depth information pertaining to the multiscopic video frame and exceeding a depth range supported by a target autostereoscopic display as measured based on a predefined perception criterion, wherein the supported depth range is smaller in extent than that of a non-autostereoscopic display, wherein the depth information comprises pixel-specific depth information represented as one or more grayscale images; determining a depth mapping function based on one or more image characteristics of the multiscopic video frame, other than depth, the one or more image characteristics including at least saliency and edges; modifying the depth information based on the depth mapping function and using: a mapping algorithm that at least partially preserves perceptible depth in the scene with a predefined degree of depth compression and by accounting for the depth range supported by the target autostereoscopic display, wherein the modified depth information conforms to the depth range supported by the target autostereoscopic display; at least partially preserving, in a third image, the saliency and edges of the multiscopic video frame, by generating the third image by operation of one or more computer processors when executing the program and based on (i) the modified depth information and (ii) at least one of the first and second images; and extending a perceptible depth range of the scene by forming multiview output for the target autostereoscopic display, including augmenting the multiscopic video frame to include the generated third image, wherein the augmented multiscopic video frame is output for display via the target autostereoscopic display.
8. A system to programmatically conform multiscopic video content to restricted depth ranges supported by autostereoscopic displays, the system comprising: one or more computer processors; a memory containing a program which, when executed by the one or more computer processors, performs an operation comprising: receiving a multiscopic video frame comprising at least first and second images depicting a scene; receiving depth information pertaining to the multiscopic video frame and exceeding a depth range supported by a target autostereoscopic display as measured based on a predefined perception criterion, wherein the supported depth range is smaller in extent than that of a non-autostereoscopic display, wherein the depth information comprises pixel-specific depth information represented as one or more grayscale images; determining a depth mapping function based on one or more image characteristics of the multiscopic video frame, other than depth, the one or more image characteristics including at least saliency and edges; modifying the depth information based on the depth mapping function and using: a mapping algorithm that at least partially preserves perceptible depth in the scene with a predefined degree of depth compression and by accounting for the depth range supported by the target autostereoscopic display, wherein the modified depth information conforms to the depth range supported by the target autostereoscopic display; at least partially preserving, in a third image, the saliency and edges of the multiscopic video frame, by generating the third image based on (i) the modified depth information and (ii) at least one of the first and second images; and extending a perceptible depth range of the scene by forming multiview output for the target autostereoscopic display, including augmenting the multiscopic video frame to include the generated third image, wherein the augmented multiscopic video frame is output for display via the target autostereoscopic display.