- Number
- 12633064
- Published
- 2026-05-19
- Filed
- 2023-09-21
- Assignee
- Lucasfilm Entertainment Company Ltd. LLC
- Inventors
- Rasmussen; Nicholas, Latta; Lutz
- CPC
- G06T3/4038; G06T19/006; G06T1/20; G06T7/20
- Verdict
- Low Notable software
- First reported
- 2026-W29 (2026-07-15)
- Source
- Google Patents · FreePatentsOnline
The keeper's note
A method of content production may include receiving tracking information for a camera with a frustum configured to capture images of a subject in an immersive environment.
Abstract
A method of content production may include receiving tracking information for a camera with a frustum configured to capture images of a subject in an immersive environment. a first image of a virtual environment corresponding to the frustum may be rendered using a first rendering process based on the tracking information to be perspective-correct when displayed on the displays and viewed through the camera. A second image of the virtual environment may also be rendered using a second rendering process for a specific display. The first image and the second image may be rendered in parallel. The second image and a portion of the first image may be composited together to generate a composite image, where the portion of the first image may correspond to a portion of the display captured by the frustum.
Background
FIELD (1) The present disclosure generally relates to generating content using one or more displays configured for operation in an immersive content production system. Specifically, this disclosure describes computer techniques for reducing latency when rendering, transmitting, compositing, and/or displaying immersive content on background screens. BACKGROUND (2) An immersive reality environment may be designed to provide a fully engrossing and interactive experience. At its core, this environment leverages advanced virtual reality (VR), augmented reality (AR), large screens, or mixed reality (MR) technologies to create a sensory-rich experience. In such an environment, users wear motion-tracking sensors enabling them to explore and engage with the environment. Within an immersive environment, users may interact with objects, environments, and other users in real-time. Immersive reality environments have found applications across various industries, from gaming and entertainment to healthcare, education, and training. (3) Immersive environments may now play a transformative role in the creation of CGI (Computer-Generated Imagery) movie sets and the way filmmakers design, visualize, and shoot scenes. These environments enable directors, production designers, and visual effects teams to immerse themselves in a digital representation of the movie's setting, characters, and special effects, enhancing the creative process and streamlining production. For example, the pre-productio
Claims
1. A method of content production, the method comprising: receiving tracking information for a camera associated with a frustum, wherein the camera is configured to capture images of a subject in an immersive environment comprising one or more displays; rendering, using a first rendering process performed by a first Graphics Processing Unit (GPU), a first image of a virtual environment corresponding to the frustum, wherein the first image is rendered based on the tracking information for the camera to be perspective-correct when displayed on the one or more displays and viewed through the camera; rendering, using a second rendering process performed by a second GPU that shares a network connection with the first GPU, a second image of the virtual environment for display on a display in the one or more displays, wherein the first image and the second image are rendered in parallel, and; compositing, using a third GPU, the second image and a portion of the first image together to generate a composite image, wherein the portion of the first image corresponds to a portion of the display captured by the frustum, wherein the third GPU generates the composite image by directly accessing the first image in a memory of the first GPU and by directly accessing the second image in a memory of the second GPU using Remote Direct Memory Access (RDMA) operations through the network connection; and causing the composite image to be displayed on the display in the immersive environment. ||
5. An immersive content presentation system comprising: a first Graphics Processing Unit (GPU) that is configured to: receive tracking information for a camera associated with a frustum, wherein the camera is configured to capture images of a subject in an immersive environment comprising one or more displays; and render a first image of a virtual environment corresponding to the frustum, wherein the first image is rendered based on the tracking information for the camera to be perspective-correct when displayed on the one or more displays and viewed through the camera; a second GPU that is configured to: render a second image of the virtual environment for display on a display in the one or more displays, wherein the first image and the second image are rendered in parallel; one or more processors that are configured to composite the second image and a portion of the first image together to generate a composite image, wherein the portion of the first image corresponds to a portion of the display captured by the frustum; and a network card that is communicatively coupled to the first GPU, the second GPU, and the one or more processors, wherein the network card is configured to access a memory of the first GPU to retrieve the first image for the one or more processors, and to access a memory of the second GPU to retrieve the second image for the one or more processors. ||
11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more Graphics Processing Units (GPUs) or processors, cause the one or more GPUs or processors to perform operations comprising: receiving tracking information for a camera associated with a frustum, wherein the camera is configured to capture images of a subject in an immersive environment comprising one or more displays; rendering, using a first GPU, a first image of a virtual environment corresponding to the frustum, wherein the first image is rendered based on the tracking information for the camera to be perspective-correct when displayed on the one or more displays and viewed through the camera; rendering, using a second GPU, a second image of the virtual environment for display on a display in the one or more displays, wherein the first image and the second image are rendered in parallel; compositing the second image and a portion of the first image together to generate a composite image, wherein the portion of the first image corresponds to a portion of the display captured by the frustum; converting a framebuffer storing the composite image into broadcast packets; transmitting the broadcast packets from the framebuffer to the display in the immersive environment without transforming the broadcast packets into a video format; and causing the composite image to be displayed on the display in the immersive environment.