- Number
- 20190342472
- Published
- 2019-11-07
- Filed
- 2018-05-03
- Assignee
- Disney Enterprises, Inc.
- Inventors
- CHAPMAN; Steven M. et al.
- CPC
- H04N21/854; G06T11/00; G06T11/65; H04N5/2228; H04N5/272; H04N21/41422; H04N21/4223
- Verdict
- Set aside real-time compositing of video content, broadcast plumbing
- Source
- Google Patents · FreePatentsOnline
Abstract
Systems, devices, and methods are disclosed for compositing real-time, real-world video with virtual objects. An electronic device includes circuitry coupled to a memory storing instructions that, when executed, cause the circuitry to receive video of a given video capture region. The circuitry is caused to receive location information and camera information from the unmanned vehicle. The circuitry is caused to obtain a representation of a location of interest corresponding to the location of the unmanned vehicle. The circuitry is caused to display the representation. The circuitry is caused to obtain virtual objects. The circuitry is caused to place the virtual objects into the representation. The circuitry is caused to render the virtual objects and the video to generate a rendered video including rendered virtual objects in at least one frame of the video. The circuitry is caused to send the rendered video.
Background
BACKGROUND
Scouting film locations may require photographers or videographers to capture a given environment without much input from a filmmaker. For example, the location may be remote, or the filmmaker may not have seen a scout's suggested location yet. The photos and videos may be captured and processed without considering the context for a scene in a movie, wasting time and money to scout the given environment.SUMMARY
Embodiments of the present disclosure include systems, methods, and devices, capable of compositing real-time, real-world video with virtual objects, as well as interconnected processors and/or circuitry, to render virtual objects to appear in a rendered video of a real-world location, including in some examples, by using video, location information, and camera information from an unmanned vehicle along with virtual object data to determine whether virtual objects will appear in the rendered video.
In accordance with aspects of the present disclosure, a server system composites real-time, real-world video with virtual objects. The server system includes a non-transitory computer-readable medium coupled to logic circuitry. The non-transitory computer-readable medium stores instructions that, when executed, cause the logic circuitry to perform a number of operations. One such operation is to receive video of a given video capture region from an unmanned vehicle via a camera. Another such operation is to receive location information and camera
Claims
1. A server system for compositing real-time, real-world video with virtual objects, the system comprising: a non-transitory computer-readable medium operatively coupled to logic circuitry and storing instructions that, when executed cause the logic circuitry to: receive video of a given video capture region from an unmanned vehicle via a camera; receive location information and camera information from the unmanned vehicle; obtain a representation of a location of interest, wherein the location of interest comprises a location of the unmanned vehicle; display the representation on a graphical user interface; obtain a virtual object that is placeable by a user into the representation; place the virtual object into the representation based on user input selecting the virtual object; render the virtual object, placed into the representation by the user, and the video to generate a rendered video comprising a rendered virtual object in at least one frame of the video; and send the rendered video.
2. The system of claim 1, wherein the given video capture region comprises a region within the location of interest the camera is capturing in a given frame of the video.
3. The system of claim 1, wherein the non-transitory computer-readable medium further stores instructions that, when executed, cause the circuitry to generate data on the location of the virtual object placed into the representation by the user corresponding to a given position in the location of interest.
4. The system of claim 3, wherein the non-transitory computer-readable medium further stores instructions that, when executed, cause the circuitry to position the rendered virtual object in the given position in the location of interest for the rendered video based on the data generated on the virtual object.
5. The system of claim 1, wherein the non-transitory computer-readable medium further stores instructions that, when executed, cause the circuitry to send the video without rendering the virtual object when the virtual object is outside a given frame of the video.
6. The system of claim 1, wherein the virtual object is a pre-designed asset.
7. A method for compositing real-time, real-world video with virtual objects, comprising: receiving video of a given video capture region from an unmanned vehicle via a camera; receiving location information and camera information from the unmanned vehicle; obtaining a representation of a location of interest, wherein the location of interest comprises the video capture region; displaying the representation on a graphical user interface; obtaining a virtual object that is placeable by a user into the representation; placing the virtual object into the representation based on user input selecting the virtual object; rendering the virtual object, placed into the representation by the user, and the video to generate a rendered video comprising a rendered virtual object in at least one frame of the video; and sending the rendered video.
8. The method of claim 7, wherein the representation of the location of interest is based on the location information.
9. The method of claim 7, wherein the camera information comprises the orientation and angle of the camera.
10. The method of claim 7, wherein the camera information is used to determine the video capture region.
11. The method of claim 7, further comprising generating data on the location of the virtual object placed into the representation that corresponds to a position in the location of interest.
12. The method of claim 11, wherein the rendered virtual object appears in the at least one frame of the video based on the location information, the camera information, and the data generated on the location of the virtual object placed into the representation when the camera captures the position.
13. The method of claim 7, wherein the virtual object is a pre-designed asset.
14. A server system for compositing real-time, real-world video with virtual objects, the system comprising: a receiver adapted to receive incoming signals; a transmitter adapted to send outgoing signals; a graphical user interface; circuitry coupled to the transmitter, the receiver, and the graphical user interface; and a non-transitory computer-readable medium operatively coupled to the circuitry and storing instructions that, when executed cause the circuitry to: receive, using the receiver, video of a given video capture region from an unmanned vehicle via a camera; receive, using the receiver, location information and camera information from the unmanned vehicle; obtain a representation of a location of interest, wherein the location of interest comprises the unmanned vehicle; display the representation on the graphical user interface; obtain a virtual object that is placeable by a user into the representation; place the virtual object into the representation based on user input selecting the virtual object; render the virtual object, placed into the representation by the user, and the video to generate a rendered video comprising a rendered virtual object in at least one frame of the video; and send the rendered video to a remote device using the transmitter.
15. The system of claim 14, wherein the representation is a map of the location of interest.
16. The system of claim 14, wherein the given video capture region comprises an image the camera is capturing at a given time.
17. The system of claim 14, wherein placing the virtual object into the representation generates data on the location of the virtual object placed into the representation that corresponds to a real-world location.
18. The system of claim 17, wherein the rendered virtual object is positioned in the rendered video based on the generated data on the location of the virtual object placed into the representation, the location information, and the camera information.
19. The system of claim 14, wherein the virtual object is designed while the unmanned vehicle is capturing the video.
20. The system of claim 14, wherein the virtual object is placed before the unmanned vehicle is capturing the video.