Depth-of-field rendering technique for immersive/virtual-production systems.
An immersive content generation system can capture a plurality of images of an object in the performance area using a camera. The object is at least partially surrounded by one or more displays presenting images of a virtual environment. The system can determine attributes of a lens of the taking camera, a first distance from the taking camera to the physical object, and a second distance between a LED display presenting a virtual image of the physical object and the taking camera. The system can determine the depth of field blur for the virtual objects based on the attributes of the taking camera and the virtual depth of each of the virtual objects. The system can apply the depth of field blur to the virtual objects and generate content based on the plurality of captured images and the determined amount of depth of field blur to the virtual objects.
FIELD (1) The present disclosure generally relates to generating content using one or more displays configured for operation in an immersive content production system. In some embodiments, the images shown on the displays may be color corrected in real-time or at interactive frame rates. The images may also have appropriate motion blur and depth of field applied. BACKGROUND (2) Immersive content systems allow for capturing the performance of actors in a performance area using one or more cameras. The performance area can be at least partially enclosed with one or more walls and ceiling of display screens. The scene can include both live actors and props and virtual elements (e.g., a virtual sun or street lamp) displayed on the one or more displays. (3) In some instances, the colors of virtual objects on a display of the immersive cave or walls may not match or may become unsynchronized with the real world colors of the actors and physical objects within the performance area. This could be a result of changing lighting patterns, light from the displays illuminating the actors, etc. As a result, content generated from images captured by the taking camera may not appear realistic to viewers due to color discrepancies between the virtual and physical objects. (4) In some instances, effects can be employed to give the perception of motion of an object on stage. For example, a vehicle can be on the stage and techniques can be employed to give the viewer a perception that the vehicl
1. A computer-implemented method of generating content, the computer-implemented method comprising: capturing a plurality of images of a physical object in a performance area using a taking camera, wherein the physical object is at least partially surrounded by one or more displays presenting images of a virtual environment; determining one or more attributes of a lens of the taking camera; determining a first distance from the taking camera to the physical object; determining a second distance between a light emitting diode (LED) display presenting a virtual image of the physical object and the taking camera; generating an image of one or more virtual objects representing one or more physical objects in the virtual environment; determining a virtual depth of each of the one or more virtual objects on the one or more displays relative to the taking camera based at least in part on the first distance and the second distance; determining an amount of depth of field blur for the one or more virtual objects based on the one or more attributes of the taking camera and the determined virtual depth of each of the one or more virtual objects; applying the determined amount of depth of field blur to the one or more virtual objects in the virtual environment; and generating the content based on the plurality of captured images and the determined amount of depth of field blur to the one or more virtual objects. ||
8. A immersive content generation system, comprising: one or more memories storing instructions; and one or more processors communicatively coupled to the one or more memories, that execute the instructions to perform operations comprising: capturing a plurality of images of a physical object in a performance area using a taking camera, wherein the physical object is at least partially surrounded by one or more displays presenting images of a virtual environment; determining one or more attributes of a lens of the taking camera; determining a first distance from the taking camera to the physical object; determining a second distance between a light emitting diode (LED) display presenting a virtual image of the physical object and the taking camera; generating an image of one or more virtual objects representing one or more physical objects in the virtual environment; determining a virtual depth of each of the one or more virtual objects on the one or more displays relative to the taking camera based at least in part on the first distance and the second distance; determining an amount of depth of field blur for the one or more virtual objects based on the one or more attributes of the taking camera and the determined virtual depth of each of the one or more virtual objects; applying the determined amount of depth of field blur to the one or more virtual objects in the virtual environment; and generating content based on the plurality of captured images and the determined amount of depth of field blur to the one or more virtual objects. ||
15. A non-transitory computer-readable medium storing instructions, that when executed by one or more processors, cause the one or more processors to perform operations comprising: capturing a plurality of images of a physical object in a performance area using a taking camera, wherein the physical object is at least partially surrounded by one or more displays presenting images of a virtual environment; determining one or more attributes of a lens of the taking camera; determining a first distance from the taking camera to the physical object; determining a second distance between a light emitting diode (LED) display presenting a virtual image of the physical object and the taking camera; generating an image of one or more virtual objects representing one or more physical objects in the virtual environment; determining a virtual depth of each of the one or more virtual objects on the one or more displays relative to the taking camera based at least in part on the first distance and the second distance; determining an amount of depth of field blur for the one or more virtual objects based on the one or more attributes of the taking camera and the determined virtual depth of each of the one or more virtual objects; applying the determined amount of depth of field blur to the one or more virtual objects in the virtual environment; and generating content based on the plurality of captured images and the determined amount of depth of field blur to the one or more virtual objects.