Method and an apparatus for editing multiple video shots
A method and an apparatus for editing at least two sequences of pictures are provided. Editing the sequences comprises modifying at least one picture of one of the at least two sequences, based on a user input, and updating at least one picture of at least one other sequence of the at least two sequences, with the modification.
1 . A video editing method comprising:
determining, for each of at least two video shots, a mosaic representative of the respective video shot, wherein at least one same object appears in pictures of the at least two video shots;
determining, for each of the at least two video shots, a transformation between the mosaic representative of the respective video shot and a common embedding space representative of the at least two video shots, wherein determining the transformation between the mosaic representative of the respective video shot and the common embedding space comprises determining color transform and warping from the mosaic to the common embedding space; and
responsive to a modification of a first picture of a first video shot of the at least two video shots, updating a second picture of a second video shot of the at least two video shots with the modification using the common embedding space, wherein updating the second picture with the modification is based on the determined color transform and warping.
2 . The method of claim 1 , wherein the modification of the first picture comprises a modification of the at least one same object.
3 . The method of claim 1 , wherein updating the second picture with the modification based on the determined color transform and warping comprises:
projecting the modification from the mosaic representative of the first video shot to the common embedding space using the determined color transform and warping to determine a projected modification; and
projecting the projected modification from the common embedding space to a mosaic representative of the second video shot using the determined color transform and warping.
4 . The method of claim 1 , wherein the common embedding space enforces a spatial invariance constraint between each mosaic representative of the at least two video shots.
5 . The method of claim 1 , further comprising determining a visibility map indicating for the mosaic representative of the respective video shot, which part of the mosaic representative of the respective video shot is visible in the common embedding space.
6 . The method of claim 1 , wherein determining the mosaic representative of the respective video shot comprises:
determining 2D trajectories of determined points in the respective video shot;
determining an embedding from the 2D trajectories of determined points to a 2D space; and
determining the mosaic representative of the respective video shot from the embedding, using a stitching technique.
7 . The method of claim 6 , wherein determining the 2D trajectories of determined points in the respective video shot is based on a 3D model of the same object projected onto each picture of the respective video shot.
8 . The method of claim 1 , further comprising:
determining a spatial relationship between each picture of the respective video shot and the mosaic representative of the respective video shot;
determining a color correction model for each picture of the respective video shot from the mosaic representative of the respective video shot; and
determining visibility information indicating, for each picture, which part of the mosaic representative of the respective video shot is visible in the picture.
9 . The method of claim 1 , wherein the common embedding space is determined using a neural network.
10 . The method of claim 1 , wherein the common embedding space is determined using local linear embedding.
11 . An apparatus comprising:
a processor configured to:
determine, for each of at least two video shots, a mosaic representative of the respective video shot, wherein at least one same object appears in pictures of the at least two video shots;
determine, for each one of the at least two video shots, a transformation between the mosaic representative of the respective video shot and a common embedding space representative of the at least two video shots,
wherein the processor configured to determine the transformation between the mosaic representative of the respective video shot and the common embedding space comprises the processor being configured to determine color transform and warping from the mosaic to the common embedding space; and
responsive to a modification of a first picture of a first video shot of the at least two video shots, update a second picture of a second video shot of the at least two video shots with the modification using the common embedding space, wherein the processor configured to update the second picture with the modification is based on the determined color transform and warping.
12 . The apparatus of claim 11 , wherein the modification of the first picture comprises a modification of the at least one same object.
13 . The apparatus of claim 11 , wherein the processor configured to update the second picture with the modification based on the determined color transform and warping comprises the processor being configured to:
project the modification from the mosaic representative of the first video shot to the common embedding space using the determined color transform and warping to determine a projected modification; and
project the projected modification from the common embedding space to a mosaic representative of the second video shot using the determined color transform and warping.
14 . The apparatus of claim 11 , wherein the common embedding space enforces a spatial invariance constraint between each mosaic representative of the at least two video shots.
15 . The apparatus of claim 11 , wherein the processor is further configured to determine a visibility map indicating for the mosaic representative of the respective video shot, which part of the mosaic representative of the respective video shot is visible in the common embedding space.
16 . The apparatus of claim 11 , wherein the processor configured to determine the mosaic representative of the respective video shot comprises the processor being configured to:
determine 2D trajectories of determined points in the respective video shot;
determine an embedding from the 2D trajectories of determined points to a 2D space; and
determine the mosaic representative of the respective video shot from the embedding, using a stitching technique.
17 . The apparatus of claim 11 , wherein the processor is further configured to:
determine a spatial relationship between each picture of the respective video shot and the mosaic representative of the respective video shot;
determine a color correction model for each picture of the video shot from the mosaic representative of the respective video shot; and
determine visibility information indicating, for each picture, which part of the mosaic representative of the respective video shot is visible in the picture.
18 . A non-transitory computer readable storage medium having stored thereon instructions for causing one or more processors to perform the method of claim 1 .