- Number
- 9911215
- Published
- 2018-03-06
- Filed
- 2016-11-04
- Assignee
- Disney Enterprises, Inc.; ETH Zürich
- Inventors
- Hornung; Alexander Sorkine et al.
- CPC
- G11B27/031; G06T7/254; G06T7/262; G06T7/97; G06T11/60
- Verdict
- Set aside video edit propagation, editing tool
- Source
- Google Patents · FreePatentsOnline
Abstract
Systems and methods of propagating edits through a video are presented herein. Decompositions of information defining frame images of a video may be obtained. Individual decompositions may include individual sets of sinusoidal functions separated into individual frequency bands. One or more frequency bands that convey differences between an (unedited) frame image and an edited version of that frame image may be identified. A subset of the one or more frequency bands that convey motion information associated with the frame image and a subsequent frame image in the video may be identified. Using the motion information, phase information of individual sinusoidal functions of a decomposition of the edited frame image, which may be included in the subset of the one or more frequency bands, may be propagated based on the motion information.
Background
FIELD OF THE DISCLOSURE(1) This disclosure relates to propagating edits through a video.BACKGROUND(2) Some applications in video processing, e.g., edit propagation, may require explicit correspondence mapping between pixels in consecutive frames. Common approaches may be based on matching sparse feature points, dense optical flow estimations, and/or other techniques. However, finding a pixel-accurate mapping may inherently be an ill-posed problem, and existing dense approaches may require computationally expensive regularization and/or optimization.SUMMARY(3) One aspect of the disclosure relates to a system configured to propagate edits through a video. In some implementations, decompositions of information defining frame images of the video may be utilized to propagate one or more edits of one or more frame image to one or more other frame images. Individual decompositions may include individual sets of sinusoidal functions and/or other information. Advantages of using decompositions may eliminate the need for computing accurate pixel correspondences between frames, e.g. extracting sparse features or optical flow. Instead, within the decomposition, image motion may be represented by phase shifts, or phase differences, between sinusoidal functions (and/or other decompositions) within one or more frequency bands of two or more decompositions. Edits to one or more pixels between modified and unmodified images may be conveyed by one or more of amplitude of sinusoidal functions w
Claims
1. A system configured to propagate edits through a video, the system comprising: one or more physical processors configured by machine-readable instructions to: obtain decompositions of information defining frame images of a video, individual decompositions including individual sets of sinusoidal functions, individual sinusoidal functions in the individual sets of sinusoidal functions being separated into individual frequency bands, individual sinusoidal functions within an individual frequency band having amplitude information and phase information, the decompositions including a first decomposition of a first frame image, a second decomposition of an edited version of the first frame image, and a third decomposition of a second frame image; identify, from the first decomposition and the second decomposition, one or more frequency bands that convey differences between the first frame image and the edited first frame image; identify, from the one or more frequency bands that convey differences between the first frame image and the edited first frame image, a subset of the one or more frequency bands that convey motion information associated with the first frame image and the second frame image; determine, from sinusoidal functions of the first decomposition and third decomposition that are included in the subset of the one or more frequency bands, the motion information; and propagate phase information of individual sinusoidal functions of the second decomposition that are included in the subset of the one or more frequency bands based on the motion information.
11. A method of propagating edits through a video, the method being implemented in a computer system comprising one or more physical processors and non-transitory electronic storage storing machine-readable instructions, the method comprising: obtaining decompositions of information defining frame images of a video, individual decompositions including individual sets of sinusoidal functions, individual sinusoidal functions in the individual sets of sinusoidal functions being separated into individual frequency bands, individual sinusoidal functions within an individual frequency band having amplitude information and phase information, including obtaining a first decomposition of a first frame image, a second decomposition of an edited version of the first frame image, and a third decomposition of a second frame image; identifying, from the first decomposition and the second decomposition, one or more frequency bands that convey differences between the first frame image and the edited first frame image; identifying, from the one or more frequency bands that convey differences between the first frame image and the edited first frame image, a subset of the one or more frequency bands that convey motion information associated with the first frame image and the second frame image; determining, from sinusoidal functions of the first decomposition and third decomposition that are included in the subset of the one or more frequency bands, the motion information; and propagating phase information of individual sinusoidal functions of the second decomposition that are included in the subset of the one or more frequency bands based on the motion information.