- Number
- 20220005279
- Published
- 2022-01-06
- Filed
- 2021-09-22
- Assignee
- LUCASFILM ENTERTAINMENT COMPANY LTD. LLC
- Inventors
- CORDES; Roger, RASMUSSEN; Nicholas, WOOLEY; Kevin, ROSE; Rachel
- CPC
- H04N13/111; H04N5/272; H04N23/90; G06T19/006; H04N23/698; H04N5/2224; G06T15/50; H04N23/695; G06T7/70; H04N13/282; H04N13/156; G06T7/20; G06T19/003; G06T15/20
- Verdict
- Low Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Immersive virtual-production system with multiple projection/tracking targets.
Abstract
An immersive content presentation system and techniques that can detect and correct lighting artifacts caused by movements of one or more taking camera in a performance area consisting of multiple displays (e.g., LED or LCD displays). The techniques include capturing, with a camera, a plurality of images of a performer performing in a performance area at least partially surrounded by one or more displays presenting images of a virtual environment. Where the images of the virtual environment within a frustum of the camera are updated on the one or more displays based on movement of the camera, and images of the virtual environment outside of the frustum of the camera are not updated based on movement of the camera. The techniques further include generating content based on the plurality of captured images.
Background
FIELD
This application discloses technology related to the fields of computer animation, virtual reality environments, and digital content generation. Specifically, this application generally relates to generating content using an immersive content production system. BACKGROUND
One method of creating a virtual reality experience can include surrounding a user with large display screens that present a virtual environment to the user. For example, an immersive content system that can be used in the production of movies or other video content can include a stage or performance area that is at least partially enclosed with one or more walls and/or a ceiling each of which can be covered with display screens. One or more cameras can be placed in the performance area and live actors can interact with props placed on the stage as well as with virtual elements displayed on the one or more displays.
Such immersive content systems can be very useful in the production of video content but present challenges that must be overcome to ensure video captured of an actor's performance using the system does not include undesirable or unrealistic artifacts. SUMMARY
Embodiments of the invention are directed to an immersive content production system. For example, immersive content (e.g., virtual reality content, mixed reality content, augmented reality content, etc.) can be leveraged as part of a system used by users (e.g., artists, engineers, technicians, directors, an
Claims
1. A computer-implemented method of generating content, the method comprising: presenting, on an immersive display, a first plurality of images of a virtual environment, wherein the immersive display comprises one or more light emitting diode displays; obtaining information regarding a position and orientation of a physical camera, wherein the physical camera is configured to capture photographic images of imagery presented on the immersive display; identifying a sub-region of the immersive display based on (a) the obtained information regarding the position and orientation of the physical camera and (b) information regarding one or more lens characteristics of the physical camera; determining a display perspective correct view, wherein the determination of the display perspective correct view is based on at least the obtained information regarding the position and orientation of the physical camera; presenting, within the identified sub-region of the immersive display, a second plurality of images of the virtual environment, wherein the second plurality of images of the virtual environment is based on the determined display perspective correct view; and capturing a third plurality of images using the physical camera, wherein the third plurality of images includes at least one photographic image of the presented second plurality of images of the virtual environment within the sub-region of the immersive display. ||
10. An immersive content presentation system comprising: one or more processors; one or more displays; and one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: presenting, on an immersive display, a first plurality of images of a virtual environment, wherein the immersive display comprises one or more light emitting diode displays; obtaining information regarding a position and orientation of a physical camera, wherein the physical camera is configured to capture photographic images of imagery presented on the immersive display; identifying a sub-region of the immersive display based on (a) the obtained information regarding the position and orientation of the physical camera and (b) information regarding one or more lens characteristics of the physical camera; determining a display perspective correct view, wherein the determination of the display perspective correct view is based on at least the obtained information regarding the position and orientation of the physical camera; presenting, within the identified sub-region of the immersive display, a second plurality of images of the virtual environment, wherein the second plurality of images of the virtual environment is based on the determined display perspective correct view; and capturing a third plurality of images using the physical camera, wherein the third plurality of images includes at least one photographic image of the presented second plurality of images of the virtual environment within the sub-region of the immersive display. ||
20. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: presenting, on an immersive display, a first plurality of images of a virtual environment, wherein the immersive display comprises one or more light emitting diode displays; obtaining information regarding a position and orientation of a physical camera, wherein the physical camera is configured to capture photographic images of imagery presented on the immersive display; identifying a sub-region of the immersive display based on (a) the obtained information regarding the position and orientation of the physical camera and (b) information regarding one or more lens characteristics of the physical camera; determining a display perspective correct view, wherein the determination of the display perspective correct view is based on at least the obtained information regarding the position and orientation of the physical camera; presenting, within the identified sub-region of the immersive display, a second plurality of images of the virtual environment, wherein the second plurality of images of the virtual environment is based on the determined display perspective correct view; and capturing a third plurality of images using the physical camera, wherein the third plurality of images includes at least one photographic image of the presented second plurality of images of the virtual environment within the sub-region of the immersive display.