Timelapse video generation
Embodiments of the present disclosure include obtaining an input video depicting a change to an image, wherein the input video has a first aspect ratio. Some embodiments compute a cost function for a frame of the input video based on a location of the change. A modified frame corresponding to the frame of the input video is generated based on the cost function. In some examples, the modified frame has a second aspect ratio different from the first aspect ratio. Then, an output video including the modified frame is generated and the output video has the second aspect ratio.
1. A method comprising:
obtaining an input video depicting a change to an image, wherein the input video has a first aspect ratio;
computing a cost function for a frame of the input video based on a relation between a location of the change and a position of a cropping window for the frame, wherein the cropping window has a second aspect ratio different from the first aspect ratio;
generating a modified frame corresponding to the frame of the input video based on the cost function and the position of the cropping window for the frame, wherein the modified frame has the second aspect ratio; and
generating an output video including the modified frame, wherein the output video has the second aspect ratio.
2. The method of claim 1 , wherein computing the cost function comprises:
obtaining metadata for the input video including the location.
3. The method of claim 2 , wherein obtaining the metadata comprises:
initiating, via an image editing interface, a drawing session; and
receiving a plurality of stroke inputs from a user, wherein the metadata includes location information for each of the plurality of stroke inputs.
4. The method of claim 3 , wherein obtaining the metadata further comprises:
assigning a timestamp to each of the plurality of stroke inputs, wherein the metadata includes the timestamp.
5. The method of claim 3 , wherein:
the metadata comprises a velocity parameter, a pressure parameter, an altitude parameter, an azimuth parameter, or a combination thereof, for at least one of the plurality of stroke inputs.
6. The method of claim 1 , wherein:
the cost function comprises a time-variant function of a state of the cropping window at a time corresponding to the frame of the input video.
7. The method of claim 1 , wherein computing the cost function comprises:
obtaining saliency information for the modified frame, wherein the cost function is based on the saliency information.
8. The method of claim 1 , wherein computing the cost function comprises:
computing a transition cost for the modified frame based on a previous frame, wherein the cost function is based on the transition cost.
9. The method of claim 1 , wherein generating the modified frame comprises:
computing a regularization term for the frame of the input video based on the cost function, wherein the modified frame is generated based on the regularization term.
10. An apparatus comprising:
at least one processor;
at least one memory including instructions executable by the at least one processor;
an optimization model comprising parameters stored in the at least one memory and configured to compute a cost function for a frame of an input video based on a relation between a location of a change to an image and a position of a cropping window for the frame, wherein the input video has a first aspect ratio and depicts the change to the image, and wherein the cropping window has a second aspect ratio different from the first aspect ratio;
a frame editing component comprising parameters stored in the at least one memory and configured to generate a modified frame corresponding to the frame of the input video based on the cost function and the position of the cropping window for the frame, wherein the modified frame has the second aspect ratio; and
a video generation component comprising parameters stored in the at least one memory and configured to generate an output video including the modified frame, wherein the output video has the second aspect ratio.
11. The apparatus of claim 10 , further comprising:
a metadata component configured to obtain metadata for the input video including the location.
12. The apparatus of claim 11 , further comprising:
an image editing interface configured to initiate a drawing session and receive a plurality of stroke inputs from a user, wherein the metadata includes location information for each of the plurality of stroke inputs.
13. The apparatus of claim 10 , wherein:
the optimization model computes a regularization term for the frame of the input video based on the cost function, wherein the modified frame is generated based on the regularization term.
14. A non-transitory computer readable medium storing code for video processing, the code comprising instructions executable by at least one processor to:
obtain an input video depicting a change to an image, wherein the input video has a first aspect ratio;
compute a cost function for a frame of the input video based on a relation between a location of the change and a position of a cropping window for the frame, wherein the cropping window has a second aspect ratio different from the first aspect ratio;
generate a modified frame corresponding to the frame of the input video based on the cost function and the position of the cropping window for the frame, wherein the modified frame has the second aspect ratio; and
generate an output video including the modified frame, wherein the output video has the second aspect ratio.
15. The non-transitory computer readable medium of claim 14 , wherein computing the cost function comprises:
obtaining metadata for the input video including the location.
16. The non-transitory computer readable medium of claim 15 , wherein obtaining the metadata comprises:
initiating, via an image editing interface, a drawing session; and
receiving a plurality of stroke inputs from a user, wherein the metadata includes location information for each of the plurality of stroke inputs.
17. The non-transitory computer readable medium of claim 16 , wherein obtaining the metadata further comprises:
assigning a timestamp to each of the plurality of stroke inputs, wherein the metadata includes the timestamp.
18. The non-transitory computer readable medium of claim 14 , wherein computing the cost function comprises:
obtaining saliency information for the modified frame, wherein the cost function is based on the saliency information.
19. The non-transitory computer readable medium of claim 14 , wherein computing the cost function comprises:
compute a transition cost for the modified frame based on a previous frame, wherein the cost function is based on the transition cost.
20. The non-transitory computer readable medium of claim 14 , wherein generating the modified frame comprises:
computing a regularization term for the frame of the input video based on the cost function, wherein the modified frame is generated based on the regularization term.